{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Topic 2: Pytorch Autograd\n",
    "================\n",
    "\n",
    "If you flag a torch Tensor with the attribute `x.requires_grad=True`, then pytorch will automatically keep track the computational history of all tensors that are derived from `x`.  This allows pytorch to figure out derivatives of any scalar result with regard to changes in the components of x.\n",
    "\n",
    "<img src=\"autograd-graph.png\" style=\"max-width:100%\">\n",
    "\n",
    "The function `torch.autograd.grad(output_scalar, [list of input_tensors])` computes `d(output_scalar)/d(input_tensor)` for each input tensor component in the list.  For it to work, the input tensors and output must be part of the same `requires_grad=True` compuation.\n",
    "\n",
    "In the example here, `x` is explicitly marked `requires_grad=True`, so `y.sum()`, which is derived from `x`, automatically comes along with the computation history, and can be differentiated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABJbklEQVR4nO2dd3iUVdqH75NeSU8gBUIJvRMQQVAQUbCjoq69LOra1111V9ey6lbL6tqWte+6on6KDURUVECl906oCYH0XkgmOd8fZyYEmCRT3pl3Jjn3deV6J/O2Z1J+88xzniKklGg0Go2m8xNgtgEajUaj8Q5a8DUajaaLoAVfo9Fougha8DUajaaLoAVfo9FoughBZhvQHomJiTIzM9NsMzQajcZvWLt2bbGUMsnePp8W/MzMTNasWWO2GRqNRuM3CCEOtLVPh3Q0Go2mi6AFX6PRaLoIDgu+EOINIUShEGJLq+fihRBfCyF2W7dxbZx7jhBipxAiRwjxoBGGazQajcY5nInhvwW8CLzT6rkHgW+llH+xCvmDwAOtTxJCBAIvAWcBecBqIcRnUsptrhjc2NhIXl4e9fX1rpzut4SFhZGenk5wcLDZpmg0Gj/FYcGXUi4VQmSe8PSFwBnWx28D33OC4APjgBwp5V4AIcQ863kuCX5eXh7R0dFkZmYihHDlEn6HlJKSkhLy8vLo3bu32eZoNBo/xd0YfoqU8jCAdZts55g0ILfV93nW5+wihJgjhFgjhFhTVFR00v76+noSEhK6jNgDCCFISEjocp9qNBqNsXhj0daeMrfZolNKOVdKmS2lzE5KsptK2qXE3kZXfM0ajcZY3BX8AiFEDwDrttDOMXlARqvv04F8N++r0Wg0/sf+5ZC/wbTbuyv4nwHXWR9fB3xq55jVQJYQorcQIgS4wnqeRqPRdC3m3wqf/Mq02zuTlvke8DMwQAiRJ4S4CfgLcJYQYjcqC+cv1mNThRALAaSUFuAO4CtgO/CBlHKrsS9Do9FofJyaYqjIhcKtUGCOBDqTpXNlG7vOtHNsPjCz1fcLgYVOW+eD/OEPfyAxMZG7774bgIceeoiUlBTuuusuky3TaDQ+zeENxx5v+gDOetzrJvh0L52OePzzrWzLrzT0moNTu/Ho+UPa3H/TTTcxa9Ys7r77bpqbm5k3bx6rVq0y1AaNRtMJscXuM8bD5v+DMx+FAO82O9CtFZwkMzOThIQE1q9fz+LFixk1ahQJCQlmm6XRaHydwxshvg+MvQkq8+Dgz143wa89/PY8cU9y880389Zbb3HkyBFuvPFGU2zQaDR+xuENkDYGBsyE4AjY/AFkTvSqCdrDd4GLL76YRYsWsXr1as4++2yzzdFoNL5ObSmUH4QeIyE0CgaeC1s/AUuDV83Qgu8CISEhTJkyhdmzZxMYGGi2ORqNxtc5vFFte4xQ22Gzob4ccr72qhla8F2gubmZFStWcNNNN5ltikaj8QdOFPy+UyAiAbZ85FUztOA7ybZt2+jXrx9nnnkmWVlZZpuj0Wj8gcMbILYnRMSr7wODIeMUKNrpVTP8etHWDAYPHszevXvNNkOj0fgThzeq+H1rolIg17sp3drD12g0Gk9SXwGleyF15PHPR6VAbTE0NXrNFC34Go1G40kOb1JbW/zeRnSK2tac3AbeU2jB12g0Gk9ia6lgL6QDUHXEa6ZowddoNBpPcngjdEuHyMTjn4/qrrbV9rrKewYt+G7y2GOP8fTTT7d7zHvvvcdTTz110vOZmZkUFxd7yjSNpmuw+GH48kE4WmW2JfY5vPHkcA4cC+lUaw+/U7Fo0SLOOeccs83QaDofDTWw4hVY+Qq8PAH2LDHbopOpOATxdmZRR1onwlYVeM0ULfgu8NRTTzFgwACmTZvGzp07aWpqYvTo0S37d+/ezZgxYwA1gHzDhg2MHj2akpISpk+fzqhRo7jllluQUk16XL16NcOHD6e+vp6amhqGDBnCli1bTHltGo1fkbcGmi1wxu8gOAz+czH8+ILZVh2joQYaayDSzrjWoBAIj4dq7wm+f+fhf/kgHNls7DW7D4MZf2lz99q1a5k3bx7r16/HYrEwevRoxowZQ0xMDBs2bGDkyJG8+eabXH/99QCsX7+eESNGIITg8ccf57TTTuORRx5hwYIFzJ07F4CxY8dywQUX8PDDD1NXV8fVV1/N0KFDjX1dGk1n5ODPgIBTboWJ98A7F8CG/8FEH5lPYcvAsSf4oBZuvSj42sN3kmXLlnHxxRcTERFBt27duOCCCwDVQfPNN9+kqamJ999/n1/84heACufMmDEDgKVLl3L11VcDcO655xIXF9dy3UceeYSvv/6aNWvWcP/993v5VWk0fsqBnyBlKITHKg+/10Qo2e31pmRtUm0V/Khk+/ujvSv4bnv4QogBwPutnuoDPCKl/EerY85AzbvdZ33qYynlH929d3ueuCcRQpz03CWXXMLjjz/O1KlTGTNmTEuP/MWLF/PRRx+1ey5AaWkp1dXVNDY2Ul9fT2RkpGeM12g6C02NkLcaRl197LmUISrEU7JbPTabFg8/0f7+qO7qTctLuO3hSyl3SilHSilHAmOAWmC+nUOX2Y4zROxNYvLkycyfP5+6ujqqqqr4/PPPAQgLC+Pss8/mtttu44YbbgCgoqICi8XSIv6TJ0/m3XffBeDLL7+krKys5bpz5szhiSee4KqrruKBBx7w8qvSaPyQI5ugsRZ6nnrsueTBaluwzRybTqTGmnLZZkgnWWXpWNfzPI3RIZ0zgT1SygMGX9dnGD16NJdffjkjR47kkksuYdKkSS37rrrqKoQQTJ8+HYCvv/6aadOmtex/9NFHWbp0KaNHj2bx4sX07NkTgHfeeYegoCB+8Ytf8OCDD7J69WqWLPHBbAONxpc4YJ0Y1WvCsecS+kFAkBoU7gt0FMOP7g5NDapVshcwetH2CuC9NvadKoTYCOQDv5FS+shvxHkeeughHnrooZOeX758OTfeeGNLj/xFixZx8803t+xPSEhg8eLFLd8/99xzAFx77bVce+21AAQGBrJy5UpPmq/RdA4O/gxxvZVo2ggKgcT+vuPhVxdBaAwEhdrf31JtWwDhcfaPMRDDBF8IEQJcAPzOzu51QC8pZbUQYibwCWC3t7AQYg4wB2jxgP2Biy++mD179hznmb/22msmWqTRdGKkVILf3059S/JgyPURp6mmCKLa8O7hmOBXF0DyQI+bY2RIZwawTkp50pKzlLJSSlltfbwQCBZC2F3FkFLOlVJmSymzk5La+UH5GPPnz2fTpk0kJraxOKPRaIyjeBfUlkDP8SfvSxkMFbmqS6XZ1BS1Hc6BY59OvJSpY6TgX0kb4RwhRHdhTU8RQoyz3rfE1RtJLy1w+BJd8TVrNG1iy2zpOeHkfcnW7JzCHd6zpy06EnxbuqaXGqgZIvhCiAjgLODjVs/dKoS41frtpcAWawz/BeAK6aKChYWFUVJS0qUEUEpJSUkJYWFhZpui0fgGB1coIU3oe/K+FGumji8s3HYk+KHdICjcax6+ITF8KWUtkHDCc6+2evwi8KIR90pPTycvL4+iIu/1kPYFwsLCSE9PN9sMjcY3OPizSse0V9cSkwEh0eYv3DZZoLa07aIrUPZHJfuX4HuT4OBgeve204hIo9F0DSxHofwAjLzK/n4hIHkQFJos+LUlgGy76MpGdHe/jOFrNBqN56nIU9vYjLaPSRkMBVu9VtBkl5aiq3Y8fFCZOl7qmKkFX6PR+Bfl1rrO2HbStpOHqGKmqsNeMckuHRVd2YhK8VpPfC34Gk1XpXAHvDsb9i012xLnKM9V2/YEv2Xh1sSwTrWDgh+dolJIG+s9bpIWfI2mqyElrPsPzD0Ddn8FC+5TC4z+QvlBEIEQndr2Mb7QU8fm4bdXeAXHF195GC34Gk1XorkJ5t8Kn90BGWPh3GdVEdPGtjqi+CDlB6FbGgS2k3MSEa86UZrp4dcUQWCISr1sDy/OttWCr9F0JfYvg03z4LR74ZpPIPtGSMuG7//slZCCIVTkth/OsZE8CIpMLL6qKVILtm20RG/BlrbphTi+FnyNpiux7TMIjoDJ90NAoBKjaY9B5SFY7Se9n8oPtp+hYyMmzWsVrHapKeo4JROOtVfwgq1a8DWarkJzM+z4ArLOgpCIY8/3ngR9p8KyZ3yj/0x7WBqgMt8xDz/Kmt/e3OR5u+xRXdh+0ZWNyCQQATqko9FoDCRvlRLAQRecvO/MR6CuFNa84X27nKHyECAdE/zo7iCboabY42bZpaa44wwdUJ+0IhJ1SEej0RjIts/UImLW9JP3pY5Suev7lnnfLmcoP6i2MQ6EdFpCJSbk4kupCq8cEXywzrbVHr5GozECKWH75yp0E9ZG1kjGWMhbo0I/vopN8B3y8HuorRlx/KOVapKVo4IflaJj+BqNxiDy10PFQRh0ftvHpI+DoxUqTdNXqchV8e5uaR0f29Jr3gTBtxVdORLDBxXSqSv1nD1WtOBrNF2B7Z+pYqUBM9s+JmOc2uat8o5NrlB+UHnuQSEdHxvp3V7zx9HSVsHBgUhhMV5ZMNeCr9F0dqRU8fvek1RBUlsk9FNzVX1lPKA9yg86Fs4B9aYQkWiS4NsapzkY0gmLgfpKj4fTtOBrNJ2d4t1Quqf9cA6onPz0sZC72jt2uUK5g0VXNqK7m+zhOxjSCYsBpIr9exAt+BpNZyfPKuCZkzs+NmMcFO+EujLP2uQKTRaVlulIho6N6O7mZOnUFAMCIhI6PBSA8Fi19XBYRwu+RtPZObwBQqJUyKYj0m1x/LUeNcklKg+BbHLew/fScJHjqC5U4bP2+v20JixGbf1B8IUQ+4UQm4UQG4QQa+zsF0KIF4QQOUKITUKI0UbcV6PROED+eugxAgIc+HdPG6OyYHxx4bbCgbbIJ2JWtW1Hs2xPxJ8E38oUKeVIKWW2nX0zgCzr1xzgFQPvq9Fo2qKpEY5sVoVVjhAapQqwcn1Q8J3JwbdhVrVtFxD89rgQeEcqVgCxQogeXrq3RtN1KdoBlnrHBR9aFWCZ1IOmLVqqbNMdP8esattOLvgSWCyEWCuEmGNnfxqQ2+r7POtzJyGEmCOEWCOEWFNUVGSQeRpNFyV/vdo6I/jp46ChytzWwvYoz7Xm4Ic6fo5Z1bbVzgp+rNrWl3vCmhaMEvyJUsrRqNDN7UKIE9MB7DWEtjtdWEo5V0qZLaXMTkpy4gem0WhOJn+DGsAR19vxc2wFWL4W1ik/4FyGDphTbdtkURXLjmbogHVIivAPD19KmW/dFgLzgXEnHJIHtP5NpQP5Rtxbo9G0gzMLtjbi+0B4PBzysUwdZ4qubJhRbWsTbVuqpSMEBCjR93XBF0JECiGibY+B6cCWEw77DLjWmq0zHqiQUpo4Tl6j6QJYGqBgi3PhHFAFWEkDoWSPZ+xyheZmlZbpyOCT1phRbWsLy9jCNI7ihfYKDiaJtksKMF+oMV5BwP+klIuEELcCSClfBRYCM4EcoBa4wYD7ajSa9ijcpjo2Oiv4AAl9YddXxtvkKrXF0Gxpf3B5W3i72rauXG3D45w7zx8EX0q5Fxhh5/lXWz2WwO3u3kuj0TjB4Q1qmzrS+XMT+kHNf5QA2TJIzMRWPBWd4vy53q62tVUpOxPSsR1ve7PwEEZ4+BqNb5PzDRzeCMGREByuiou6DzXbKs+Tv16JtTMLtjZsVbkleyDNB+okq6yCH9Xd+XOju0PBVmPtaQ93Qjql+4y25ji04Gs6L3Xl8OUDsGne8c8HBMH5z8Ooq00xy2vkr1fhHGEvSa4DWgQ/xzcE35Zl42h/+da0rrYNCDTWLnu46uH7Q0hHo/FJ9i2F+beq2O3pD8CEu1Q8u74CFvwaPr1ddZE881HnMlj8hcZ6KNgGE+5w7fz43oBQgu8L2EI6US6GdGzVtq6EhJylky/aajS+xZHN8O5lKmf75q9VCMdGRDz84kP48rfw4z+gMh9mzXXNC/ZlCrdCcyP0GOna+UGhKgXSVwS/qkClLYZEOH9u62pbbwh+XbkKHzoypKU1YTGq4K3J4njTNSfphK6NpktTXwEfXKu8qxsWHi/2NgKD4Nxnlee/+QPY+aXXzfQ4hdvVtvsw16+R0M93BL+6wDXvHrxfbVtX7nw4B459IvBgT3wt+JrOg5Twya9Ugc7st9uP9woBk3+r8s2/+p0KgXQminZAYCjEZbp+jcQstWgr7RbFexe3BN/L1bb15c6Hc6BVP51yA405Hi34ms7DTy/Aji/grCeg5/iOjw8Mhhl/g7L98NM/PW6eVynapQTbnUXKhH7QUG1OP/kTqS5wPRzj7Wpblz18zzdQ04Kv6RyU7oVvn4BBF8D42xw/r8/pMPhCWPaMas7VWSjaAUkD3LtGQl+19YWwTpUbHr63q23rypwvugIt+BqNw3z9KASGKI/d2QXY6U+q7eKHjbfLDBpqVVgr0V3Bb5WaaSZHq6GxxnXBB+9W27od0tGCr9G0zYGfYPtncNo90M2FMQuxPVX64rZP1CcFf6dkNyDd9/C7pat1ALMFv6XK1oWiKxverLZ1NaRjO8eD1bZa8DX+TXMzfPV71WPlVBdzzgGyb1Sj/da9Y5xtZlG0U23dFfyAABXWMbuJWksOvgtFVzaiUtScWU/T1Kg+jWgPX6PxAJs/VBWl0x51LUfbRrdUyDob1r+r/mn9maKdIAIhvq/710roa76HbwvFuNJWwUZEAtSWeD7jqKVxWqzz54ZEKadDC75GYwfLUfj2j6p9wLDZ7l9vzHVQUwi7Frl/LTMp2qGE2tnCH3skZKn+Lk0W96/lKu5U2dqISICmo9BQY4xNbdHSVsGFRVshPF5tqwVf479snAeVeXDmI8a0R+h3lirSWfu2+9cyk6KdkNjfmGsl9FMVu+UHjLmeK1QXQECwayJqwzZ9qrbEGJvawtW2Cja04Gs0dmhugh+fV60D+kwx5pqBQTDyKtVd019TNC0NauE5aaAx12vdNdMsqgpU/N6dN3VvCb47IR1QbxS68EqjOYHtn0HpHjjtXmP74Iy+BpCw4V3jrulNSveAbPKA4JsYx3enytZGZKLaag9fo/EzpITlzykxGnS+sdeOy1SfGNb9R2UA+RtFO9Q2yaCQTkS8Ei+zBd+dlEzwoofvRgwftOBrNCexZ4kaaDLxbs/0Nx9xpVob8LUh3o5QtAsQarHVCIRQQ83LPDuYo12qC9xLyQT1xgXeC+m4OiXM1wVfCJEhhPhOCLFdCLFVCHG3nWPOEEJUCCE2WL8ecfe+mi7M8udU3v3wyz1z/ayzVFrjLj/solm0A+J6uZeieiKxGeataTRZVB97d1IyAUJj1O/UGyGdkGjX2xv7uuADFuA+KeUgYDxwuxBisJ3jlkkpR1q//mjAfTVdkfwNsH8ZnPor1bPdE0TEQ89T/bNtctFO91sqnEhMBlTkmdM1s6YIkO57+AEBKqxTU2yIWW3iapWtjfBYaKxVi+8ewG3Bl1IellKusz6uArYDae5eV6Oxy6q5arjE6Gs9e58BM6Bwm+qk6S80WVSs3d0K2xOJ7QmWOs+LpT1sLY3djeHDseIrT1JX5vqCLRw710NevqExfCFEJjAKWGln96lCiI1CiC+FEEPaucYcIcQaIcSaoqIiI83T+DvVRaqyduSVrsdIHWXADLXd6UdFWOUHVHGRURk6NmIy1LbioLHXdYQqA4qubEQkQG2p+9dpj/py9zx8D7dXMEzwhRBRwEfAPVLKE0e2rAN6SSlHAP8EPmnrOlLKuVLKbClldlJSklHmaToD695Sc2nHzfH8vRL6quIlf4rjG9VD50RirYJvRhzfiCpbGxHx3lm07eyCL4QIRon9u1LKj0/cL6WslFJWWx8vBIKFEIlG3FvTRWhqhNWvQ9+pxgtaW/Q/B/b/6PHB0oZRvEttEw3K0LHR4uGbKfhuxvBB5eJ7Y9HWrZCOZ6deGZGlI4DXge1SymfbOKa79TiEEOOs9/XwT17Tqdj+uWpvO+4W791zwEzVViDnW+/d0x1KctR0J6PDXeGxaoC4WR5+eJwxC/QRCVBX6tn6iroyNz1867kecjKMGI0+EbgG2CyE2GB97vdATwAp5avApcBtQggLUAdcIaUvDMrU+A0r/wVxvSFruvfumTEOwuNVM7Whs7x3X1cpyTlWGWs0MRnmePhVR9xPybQRkQCyWXnPtrx8I2msB0u9T3v4bgu+lHI50G5tu5TyReBFd++l6aIc3gi5K+DsPxnTJM1RAgKh/9kqPbPJ4nputbcoyTm22Gw0ZuXiVxcaE86B46ttPSH4NpF2p8mbP8TwNRqPsvo1CApXjc28TdZ09Y+cv97793aGunKVs+5RD9+ELJ3qI8akZILn2yu42zgNIDhcdQbVgq/pktSVw6YPYfhl7v0juUrvyWq7f6n37+0Mtm6WRrVUOJHYDCVC9Scm4HkQKT3j4XuqnsDdxmng8Z74WvA1vs3G91TRT/ZN5tw/MhGSB8O+Zebc31Fszc086eGDd+P49RUqJm5ESiZ4wcO3NU6Lde864bFa8DVdEClVKmZaNqSONM+OzEmQu9Jj5e6GULJb9YqJy/TM9WN7qq034/i2GbRGLtqCF0I6bsTwQXn4HhpkrgVf47vsW6qEbOzN5trRe5Lqb+LL3TNLclTTNCPGGtrDDA+/xib4BhVghkRAcITnBN+IkA7okI6mi7L6NeUtDbnYXDt6TQSEatrmqxTneC5+DxCZBIEhUO7FhVubhx9pUAwfPNtPx93WyDa04Gu6HJX5sGMBjLoGgsPMtSUiHroPVZ84fJHmZjXpylPxe1DpsDHp3vXwW0I6BsXwwbPtFerKVBtmd2c0aMHXdDnWvKmKZLJvNNsSReZkyF2limt8jap8FXJK9KDggwrreDOGX1Oo1iXcjYm3xpMefn05hBtQ5Tz+V3D5f92/jh204Gt8D0sDrH1LDSKJ7222NYrek1QnyrzVZltyMp7O0LER6+Vq2+pCFUoystguwoP9dOrKjXlzShoAPU9x/zp28PHSQdeY/erPHLU0OXawEC1lwkKokuEAIQgQAiHU48AAQUCAIMj6FRwYQFCgICQwgJCgAEKDAgkLDiA8OJDwkEAiQoKICgsiKjSQmPBgYsKD6RYeTFxECMGB+j22Q7Z/pry7sb802xIAGpua2RY4mGEEsPiLD/koJpwGSzPNUtIvOYqhqTEMT4+hX3IUwsiB6o5SvFttPRjDL69t4HBdLIOqC5jzxo+kJsaSmRDB2N7xDEn1UKtqI3PwgdoGC0X1YaRUFHLH22s4Z2h3zhnanahQg2TQ3cZpQHOz5L8rD7D+YDnPzh5h+N9TpxT82IhgGpo6jqNJCbLlsWx5rlnKlq2luZmjFkmThKbmZixNksamZizNkgZLc8tXvaWJxqaO2wPFRgSTEBlCSrcwuncLo3tMGGlx4WTERdAzPoK0uHD9prD6NZVe2G+aqWZszqvg7Z/3s2DTYeoam/g0JJPupavJbbqU0KAAmiXMW5VLXeN+AAZ2j+a6CZlcODKViBAv/muV7IGQKOMqUltxuKKOP3yylSU7CrhYWHgmBJrKc/lwfxU1DcqpmjUqjQdmDCSlm8FrLTXGCH5Ts+Svi3bw1k/7mSPr+E1wLTn5JfxmewEPf7KZ84en8scLhxIe4mbsva4Mkge5fHpOYTW/+3gTq/eXMSkrkbrGJsP/jjql4M+9NtuU+zY2NVPX2ETNUQs1Ry1U1VuorLdQUddIeW0DpTUNlFQ3UFx9lILKelbsLaGw6iiW5mNvFEEBgl4JEfRLjqJ/SjSDe3RjcGo3MuIiCAgwwXv0Nke2wMGf4awnvNs3pxXf7Szkn9/uZt3BciJCArlwZCqT+yfR78AMItfNZdGvsltmxjY1S/YWVbNiXyn/W3mQ3328mT8v3M5dZ2Zx/YRMgrzx5l2yW/XvN9AblFIyf/0hHv1sK5Ymya2n92VWfAAsfJXXL0xG9plCUdVR3vppP68t28eirUe4d1p/bp7U2zivtLpIFb25QX1jE3fPW89XWwuYNTqNi6OGw6oP+e72YawrC+PjdYd4b9VBDpTW8sb1Y93z9uvKXfbwP1idy8OfbiE8OJCnLxvBJaPTPPJpsVMKvlkEBwYQHBhAt7Bgh89papYUVNaTW1rLwdJa9hXXkFNYze7Car7eVoDtvaBbWBAje8YxMiOWMb3iyO4VR6RRH0V9idX/hqAwGHW112+9u6CKJxds54ddRfSMj+DR8wdzyZj0Y7/P0DNgzUsqjt/ndAACAwRZKdFkpURz9Sk9WXOgjJe+y+HJBdv5ZMMh/nzxcIale3g6V0kOpI0x7HL1jU3c98FGFmw+zNjMOP5+6QgyEyOhzOrBl+cihCC5Wxj3nzOQK8b25I9fbOOphdvJr6jjkfMGuy9WUioPP9L1HPzSmgZufns163PL+cN5g7nptN6w7QCsAlFbypheQxnTK55T+iRw7/sbuPb1lbx14zin/n+Ps7e+3KUY/toDpfx+/mbG90ng2ctHkBztuay0TqgY/kVggCA1NpzU2HBO6ZNw3L76xiZ2FVSxNb+STXkVrD9YxotLdtMs1SeB4ekxnNo3gdP7JzO6Z6x3vElPUl8Bmz6AoZd6ppthW7dtbOK5r3fx2vJ9RIQE8vC5g7j21ExCgk74eWaMVdvclS2C3xohBGMz43nz+rF8ueUIj362lQtfWs5dZ2Zx19Qsz3xCsxxVufHDrzDkcrUNFua8s5blOcU8cM5A5kzuQ6DN7m5pIAJOWrjtmRDBv68dwxNfbOeNH/dRe7SJP80aduw8V6gvV9PNXAzp1Dc2cd0bq9hZUMVLvxjNzGE91I6Wattj/XQuGJFKSGAAd763jqv+vZIPbjnV+fBOY52y18m2CqU1Ddzxv/Wkxobz8tWjXXuzcQIt+D5MWHAgw9NjGZ4ey5Xj1HPVRy2sP1jGz3tKWLG3hFd/2MtL3+0hOiyIyf2TOHtId6YMSCLaw384HmHdf1R64TjvVdZuyivn1x9sJKewmsuzM7j/nAEkRLUxbCM8DpIGKcFvByEEM4f1YGK/RB77bCv/+GY3W/MreXb2CON/L6X7VPqqARk61Uct3PjmatYcKOXpy0Zw6Zj04w8IDIboHnZTM4UQ/OG8QUSFBvLCkhzqLU08N3uk629ybubgP/rpVjYfquDf12Zz1uBW12ijvcI5Q7vz8lVj+OU7a3hywTaeuniYczd0ocq2uVly7/sbKKlu4ONfTfC42IMWfL8jKjSISVlJTMpSH3Ur6hr5MaeY73cWsmRHEQs2HSYkMICJ/RK4YGQqZw02MAvBkzRZ1JCTnhMgdZTnb9cseXFJDi8s2U1SVChv3ziO0/s7ED7oeQpsma+KnTpYY4gJD+bZ2SMYlhbDUwu3c/HLP/HatdkqPGIUJdYMHTdz8GsbLFz7+ko25lXw/BWjOH9Eqv0DYzKgIs/uLiEEv54+gNDgQP7+1U56xkdw33QXx1G2VNk6H9J5f/VB3l+Tyx1T+h0v9tBK8E8eZn7W4BTmTO7D3KV7OWNA8snntocLjdP+tXQvP+wq4smLhjI0zcNhPyt+oASa9ogJD2bmsB7MHNaDpmbJ+oNlfLX1CAs3H+He9zcSFryZswZ359Ix6ZzWL9G9j9meZOcC1W/97Kc8fquCynrumbeBn/eWcNHIVB6/cCgx4Q56VxnjVY1A0XZIGdLh4UIIbjytNwN7RHP7u+u45JWfePvGccb9g9tSMuP7unwJS1Mzt7+7jg255bx81WjOGdqj7YNj0jrsKfSrM/pysKSWfy7JISslmgvaevNoj5Y+Os6FdDbnVfCHT7cyKSuRe8/qf/IB4dZQYRu5+PdN78/y3cU88NEmRqRPItnRzKOWtgqxDh1+oKSG577Zxcxh3bnqlJ6O3cMAjBpifo4QYqcQIkcI8aCd/UII8YJ1/yYhxGgj7qs5nsAAQXZmPA+dO5hl90/h/249lcvGZLBsdxHXvbGK0/66hGcW7ySvrNZsU0/m55chthcMPNejt1m6q4iZzy9jQ245T182gn9cMcpxsQc19hA6DOucyIS+iXx02wTCggO5cu4KVu41qPinJEeFWcK6uXS6lJKH5m/hu51FPHHR0PbFHlQcvzK/3bmwQgieuGgoYzPj+O2HG9mUV+68YS6EdGqOWrj9f+tIjAzh+StG2XduAoOUKLch+KFBgbxw5UhqGyzc9+FGmps7TrUGWk27iu3wUCklj322lZDAAB49f4hXazeMGGIeCLwEzAAGA1cKIU7MpZoBZFm/5gCvuHtfTfsEWMX/iYuGsvL3Z/LyVaMZ0D2al77LYfLfvuOmt1bz3Y5Cx/+gPcmhtWqE4Sm3ut+HpA2amiXPfr2L695cRWJUKJ/fOfHkGLUjxPdRYYaDzgk+QJ+kKD689VSSu4Vy7Rur+HZ7gfP3P5HiXW7F75//djfvr8nlzqn9uOqUXh2fEJOuFidr2x8iEhIUwCtXjyExKpQ576yluPqoc4ZVF0JAkFMx8ScXbCO3rJbnrxxFfGQ7XUMjEtodgtIvOVo5TbuLeX+Ng5XFTrRG/npbAd/tLOKeaVnG1y50gBEe/jggR0q5V0rZAMwDLjzhmAuBd6RiBRArhOjAlXCDynyoOHTsqzJfDUOuLoSaEvXLOVqtSvi7wCz10KBAZg7rwVs3jGPZA1P51Rn92JhXwQ1vrWbqM9/z5o/7qKpvNM/AFa9ASLTHUjFLqo9y3RureOHb3cwalc4nt0+kX3K0axcTAjJOUW9QLpAaG86Ht05gQPdobv3vWhZtOeKaHaD+dot3Q6Kd0IUDfLAml398s5tLRqfza3vhD3t0S1PbNuL4rUmMCmXutWMoq23g7nnraXLGuahxrq3Ct9sLeG9VLrdM7svYzA4yvBzop3P1KT05pXc8f1q4ncJKB/onObhoW9fQxOOfb2NAiirS8zZGxPDTgNZvg3nAiY0g7B2TBhw24P4n888xKtvDEUSAyvsOjoDQaPUVFqP+KCITVWvWmDT1hx7bU30F+mEGjJW02HB+c/YA7jozi0Vbj/Dmj/t4/PNtPLN4F5ePzeCGiZmkx0V4z6DKfNg6H8bNcTks0R4r95Zw17z1lNU28tdLhjE7O8P9j9A9x8OOL6CqAKKdzyKJjwzhvzefwnVvrOKO/63jhStHHUsbdIaaYiU0ic63VPgxp5jff7yZ0/ol8pdLhjn+M4mxCn7lIUjrODI7JDWGP144hAc+2szz3+52/I3FibYKJdVHeeCjTQzsHs29Zznws4hM7LAJnBCCP88axjnPL+Oxz7fy8lUd1DnUlQECQtv/G375+xwOldfx/pzxplTUGyH49v5STnwrd+QYdaAQc1BhH3r2dHExY+bT0Gw5dhsp1ba5ydo7wQLNjeqjaWO9GqPWUAMN1XC0Sn0CKNiqPrbaVt9tBASpsv+ELNUyt/tw6DFCvRGY0UfFRUKCArhgRCoXjEhlQ245byzfx1s/7eetn/YzY2h3bpnc1/MFQwA/v6TSCsfNMfSyzc2Sl7/P4dmvd9ErIZI3rh9rXM+XDKs/k7sSBl/g0iW6hQXzzo3juPGt1dz53noam5q5cGSacxcp3qW2Tgr+ziNV3PqftfRJiuTlq0c7JzzdrGGwikMOnzI7O4NV+8r455LdjOkV51g2VHWhQ33wpZT87uPNVNZZ+O/NpxAa5EBIMCIe8jd0eFifpCjumtqPpxfv4uttBe1n7dSVK0exnU8kOYXVvPrDHi4amXpSzY23MELw84CMVt+nA/kuHAOAlHIuMBcgOzvbtXjLqKtcOs0ulqNQdVj9gZcfUH1LSnKgaCfs/kqJFaiFs57j1bCMvlNVqbufMDIjlheuHMWDMwby1k/7eW/lQb7YdJhT+yQw5/Q+nNE/yTMLS7Wlqg3y0EsN7Yp5pKKe33y4keU5xVwwIpU/zRpmbGpqjxEQGOqW4ANEhwXz1g3juOnt1dzz/gaONjYze2xGxyfaaEnJdDykc7iijhvfWk1YSCBv3uBCVWlkonrtlR2HdGwIIXjyoqFsOVTBPfPW8/mdp3X8KbKmyKEsqP+uOMDibQU8NHMQA7s7+AkxIkE5c1J26KTNmdyXzzce5pFPtzC+T3zbdRT15e0u2KrF8c1EhATx8HnutYtwByP+C1YDWUKI3sAh4ArgFycc8xlwhxBiHircUyGl9Ew4x2iCQpVHH5cJTDx+X2MdFGyD/HVwcIXqAbN1vtoX1xuypsOg86HXBI8tRhpJamw4v585iDun9uO9VQd5Y/l+bnhzNQNSovnl5D6qIvHE6lN3WPEKNNbApF8bdskvNuXz0PwtNFia+fOsYVwx1oAQzokEhapwhpOZOvaIDA3izevHcct/13L/R5uobbBw/UQH3/yKd0NQ+DGvu6PDq49y1WsrqahrZN6c8aTFhjtvsBDQLdUpDx8gPCSQV64ezYUv/cgv31nLR7ed2nZjMCkdCulsza/giQXbOWNAkmqb4CgRCerTfUMNhEa1e2hIUAB/uWQYl7zyE499to1nZo+wf2AHfXT+b20eK/eV8pdZw0hsq7DPC7j93yultAB3AF8B24EPpJRbhRC3CiFutR62ENgL5AD/Bn7l7n19guBwSB8D434Jl74O926Fu9arkFJif1j3Drx9Hjw7GBbeD/nrzbbYIaLDgpkzuS9L75/C05eNQAj4zYcbmfS3Jbzy/R4qag1Y4K2vhFX/goHnudVh0EZx9VHunreeO/63nt6JkSy8exJXjuvpuZS3jFNUWKCxzu1LhYcE8u9rxzB9cAqPfb6NF5fsbune2i7Fu1WGjgMLmxW1jVzz+iryy+t44/qx7tUBxKSrGL6T9EmK4oUrR7HjSCW//XBT26+xrkyFXNsJ6dQctXDn/9YTFxHMM5eNcK6it4Nc/BMZ1TOOO6b046N1eSzY1Iaf2k4fndKaBv60cDvZveKYne3EJzgPYIi7JqVcKKXsL6XsK6V8yvrcq1LKV62PpZTyduv+YVLKNUbc1+cQQqXtjfslXPUB3L8HLn0D0rNVsc7cM+DVSar9b32l2dZ2SEhQAJeOSefLuyfx9o3j6JccxV8X7WD8n7/lkU+3sLeo2vWLr35N9c6Z/Bu3bGxulry36iBnPvMDCzcf5p5pWfzfrafS28hqVntknKJEyaA38dCgQF66ajQXjUzl6cW7eOCjTTRY2s51B1QM34EK24q6Rm54axV7Cqv51zXZjOvtZp+ibmlOe/g2pgxI5nczBrJg82FeXJJj/6Dq9ouubOGR/SU1/OPyUW23wmiLNtortMedZ2YxIiOW38/fzOEKO2/ydeV2QzpSSp74YhtV9Rb+NGuY6R1v/bzblo8TEglDL4Er3oXf7FKePxIW3AfPDYHFDzuU3mY2QghO75/EuzePZ+Fdkzh3eA/mrcpl6jM/cM3rK/lq6xEsTR2IU2saatVibd8z3WqjsGJvCZe8+hO/+3gzA7tH8+Xdk7hnWn/vNJGzFWAZOAErODCAZ2eP5K6p/fhgTR7XvbGq7U9TlqNqTamD+H1uaS2XvPITm/IqeOHKUY4tmHZETJpa12p2cMjQCfxyUh9mjUrjma938e7KAycf0E6VrZTS2ok0n3um9efUvi4sftoEv+7k9gptERwYwPOXj6SxqZn7PrBTkFVXZjek89ZP+5m//hC/mtKP/ikupgIbiBZ8bxEeqzz/W5bBzUvUcI+fX4bnR8D829RisB8wOLUbT182gh8fnMp9Z/Unp7CaW/6zlol/XcJfF+0gp9ABr3/NG2rRbPJvXbJh/cEyrnl9JVfMXUF+eR1PXzaCeXPGu55b7wqRierTXO4qQy8bEKD60Tw7ewRrD5Rx3ovL+HmPHU+0dK+1aVrbGTprD5Ry0Us/UlR1lHduGsc5Qw0akNItDWSTqm1xASEEf5o1jKkDk3lo/hbmLj3hb7+lj87Jgv/04p28vnwf10/I5M6pLhac2Tqx2umn0x6ZiZE8ev5gftpTwkOfbD5WV9DSGjn2uOO/3V7AE19sY/rgFO4503PTyJxB99LxNkKouP9lb6q2tj+/DGvfhE3vw4grlAj6yhzXdkiKDuXOM7O47Yy+fLujkA9W5zJ36V5e+X4Pw9Nj1Pi4Id3pk3TColhdOSx7GvpMgV6nOny/uoYmPt+Uz7srD7Ixt5z4yBAePncQV4/vRViwSQvi6WNh7/cOZXs4y6zR6fRKiOTXH2zgyn+v4JrxvXhwxsBjMxDaScmsbbDwrx/28soPe+gRE8Yb14+l74m/B3eIsS4SVx46lpfvJGHBgbx69Rju/WADf1q4g+qjTdw7LUutudgJ6TQ3S57/djcvfbeHK8dl8Oj5bvTcdyGkY2N2dgYHS2t56bs91DY08cxlIwiy1KpU71Ye/rb8Su58bz2DU7vxjyvc6BpqMFrwzSS2J8z4C5x2L/z4D+X5bvpA5aRP/o1Xe8K7SlBgAGcP6c7ZQ7pTWFXPp+vz+WJTPn9btJO/LdpJn6RITu2TwPg+CYzNjCdl1XOIunI464/tXldKyZHKepbuKuL7nUUs311M1VELWclRPHr+YC7LzjC/C2j6WPVGXZGrfpcGM6ZXHIvunszTi3fyxo9qqtQVYzO4fGwG6S1zbI95ufWNTSzYdJi/f7WTI5X1nDu8B09cOLT9NgOu0Lra1hbacoGQoABeuGIUkSGBvPDtbr7fWcivz+rP6dWFiIDglkXQn/eU8OSCbWzNr2TWqDSeusiJQjF7hMWogksnPXxQn05+e/ZAIkKC+PtXO6lraOIv0+KJBwiPpa6hiXdXHuDl7/fQLSyY168b691xlx0gHMoGMIns7Gy5Zk3nXN+1S+Vh+O4pWP9fVXV6+oMqDOSHlb355XV8tfUIS3cVsXp/GdVHLaRSzHdh9/Fz2CQ+6/MoseEhxIQHExESSH1jEzUNTVTUNbCnqIbdBVWUWePX3buFccaAJGaNTmdsZpw5g8LtcXgj/GsyXPI6DLvUo7dae0ANv/l+VxEAb8W8znDLJl4d/TkIWHegjI25FTQ0NTMiPYY/nDeY7I5aDLhKXRn8NROmPwkT7nT7cs3Nkv9bl8fz3+zmUHkdr8e+yRjLBv7Q90OKqupZsbeUtNhw7j9nAOcPTzXGW/5rbxhyMZz3rMuXsFWpDxQHWRT6IO/2epLnDg2iuPooE/om8PgFQ8gyIW4vhFgrpbQ759V33no00K0HXPgijL9NLeh+9TtY/x+Y+XfIPM1s65wiNTacGyb25oaJvbE0NbM1v5LIL+8k8DD8X7frWLenhIq6xpZB2KA8vujQIPokRXLO0O5kJUczoV8CA1KifUfkW5M8RLXkyFvtccEf0yuON28YR15ZLe+vzqXHqlxymnvw5k/7aWqWDE2L4fqJmWoCWlaSZ0MIYbEQHOlyps6JBAQIZmdncNHINN5fk0v0N6Ucae7G1kMVhAQF8NuzB3DTab2NDd050E+nI26Y2JtxvePZtXIRbISv9tbTv2cUL/1ilGmVtB2hBd8XSRkCV38MOxfClw/CW+fCsNlw9p8gyoAsCy8TFBjAiOBcyP8cJtzBi9MvatlnsQ5+DwsONKW3iFsEBkHqaEMzdToiPS6C+87qD2uPwKjL2TnzHCzN0rs/OyFU7N6JaltHCAkK4JrxvWCjBaL6suSqMwy9/nFEJDiVpdMWQ1JjGDIwAjbC27edhUgd6b5tHsTP/sO6EEKo3vC3r1QLuVvnw0tjYcP//K/DZ3MzLPiNip2ednxVbVBgANFhwf4n9jYyxsLhTaonk7eoLoCjlZDYHyGEOT87N3LxO6S6yKE+Om4REe9SDN8u1tbIwoUB5t7GT//LuhAhETD1Ybh1ucq5/uQ2+M9FUGYnf9lXWfO6aid89p/8YiHaKdLHqgKswxu8d0/bgq2bYw3dIibNpWrbDmluVnn4Lg4vd5iIeLdDOi04MfzEbLTg+wvJA+GGRXDuM5C3Bl6ZoKpV25k85BOU58I3j6k0zJEntljqBKQbX4DVIS0pma71wTeEbukqfdLSYOx168tViqPHBd8awzfi03Jducr6CTG/sKojtOD7EwEBMPZm+NXPyrNccB+8cwGU7TfbMvtICQt+rQqEzv+HX7WPdpioJNVYz+ACrHYp2qkWTaNdmBVrFDFpgIQqu01vXafaOgXMheHlTtG6gZq71JerhWwHh7WYie9bqDmZ2J5wzXy44J+qgdfLE2D1674X29/0AexeDFP/YO022klJH6s8fG/9/Au3qYZzZgpMSy6+wWGdKmtzsm4efjNzsoFau7TRR8cX0YLvrwgBo69V3n7GWOVJ/+ciVb3rCxRshS/ugYzxcMotZlvjWdLHKaHyREz7RKRUP9sU83qqA8dX2xqJrV1DtEFtINrCjWrbk2ijj44vogXf34nNgGs+gXOfVbH9lyeooSJmevt1ZTDvKjXubfbbfjELwC0yxqqtAf3xO6S6QKUTJnc8HMSjODHb1ilsHn6UlwTfgNTMjoaf+BJa8DsDQsDYm+C2nyBtlPKs/3OxObH95ib46GYlBLPf8byn5gukDFUFWN6I4xdstd7TZA8/NEql2XrCww+LUdlpnsTFBmp26WD4iS+hBb8zEdcLrv3M6u2vhpdPVW2IXWxj6zRSqgrhnG9g5t+g54mz7DspgcGQNkZNPfM0hdvU1mwPH1SmjtEx/Mp87yxGGxnS0R6+xjRs3v7tK6H3ZPjq9/DamZC31rP3bW6Ghb+BFS/DuFtgzA2evZ+v0XM8HNkMR90YCuMIBdsgKgUifaB03wPVtlQd8c6nQjcaqB2HlNZFW98vugIt+J2XmHS4ch5c+qbyml6bCp/cfqz1rJE0WeDT21VdwIS7YMZfO2cKZntknKJ6xB/y8Btr4VZINjmcYyMmQ9VZGEnVEYjuYew17REQqMIw7nr4DdXq994VQjpCiL8LIXYIITYJIeYLIWLbOG6/EGKzEGKDEKILtb80GSFg6Cy4c60S4k3vwz/HwHd/bikHd5vKw/DeFbDxf3DG71Xb464m9qBSMxGeXbhtblI5+Ck+EM4BlTBQXw5Hq4y5XnMzVHvJwwdDGqhRV6a2XSSk8zUwVEo5HNgF/K6dY6dIKUe21bZT40FCo2H6EyqFs/dk+OEv8Pxw+OFvUFPs2jWlhA3vwcunwP5lanzjGQ90TbEH9Q+fPMizgl+6Fyz1vuXhg3Fefm2JqrL1hocPBgl+udp2BQ9fSrlYSmmxfrsCSHffJI3HSMxS83VvWQq9TlO9958ZCB9eD3uWQFMb81NbYzkKWz5SHTw/uRWSBsGtP6q+/V2djFMgd7Xn2l3YMnSSB3nm+s5iG/pSYZDg26p2u3lR8G0euqv4UR8dMLY98o3A+23sk8BiIYQE/iWlnNvWRYQQc4A5AD17Gj9FSAP0GAFX/g8Kt8Pat2HTPNWNMzhSZdb0mgCxmeqPOCxGfQooyYHinbBjocpdjukJM/6mWj109jx7R+k5Xo2rLNrumbBL4TZAQNJA46/tCi0evkHFfi1FV94S/DjIX+feNfzMw+9Q8IUQ3wD2gmoPSSk/tR7zEGAB3m3jMhOllPlCiGTgayHEDinlUnsHWt8M5oKaeOXAa9C4SvIgNWJx2mOw+yvYtwwO/AhLnrR/fHg89J4Eo69TzdD8oHeIV7GN+zu4wjOCX7BVDU73dI66o0SlQGCIgR6+tejK2zF8d2YSt3j4/pGl06HgSymntbdfCHEdcB5wpmxjXqKUMt+6LRRCzAfGAXYFX2MCwWEw+EL1BepjbnWh8l7qK9Q/RkIfv/mjNo243qqPe+5KlRprNIXbzS+4ak1AgKq4NSqGX3UEEOqNxBu0bqAW6uKQd5uH3xVCOkKIc4AHgNOllLVtHBMJBEgpq6yPpwPtT7DWmEt4nBZ3VxBChcQ8UYDVUKsWbYddZvy13SE2w1gPPzLJezOcWzdQc1nwy0AEQoiL53sZdz+TvwhEo8I0G4QQrwIIIVKFEAutx6QAy4UQG4FVwAIp5SI376vR+CYZ46H8wLF4tFEU7QCkb3n4oNZyjPLwKw97txWHEdW2tipbP8lOc8vDl1LaHbljDeHMtD7eC4xw5z4ajd/Qc7zaHlwBQy4y7rq+1FKhNbEZKnfechSCQt27VtVh7y3YQivBd6Pa1o/66ICutNVojKX7cJXttH+5sdct2ApBYRDf29jruostU8eIrpneaqtgw9ZAzZ2OmX7URwe04Gs0xhIUAr1OhX0G5yTkrYbUUb6XAhtrE3w3wzpNjVBTZJKH70ZIx4/66IAWfI3GeHqfrmoWKg8bc73GejXZLH2sMdczEqOqbasLAOm9oito1UDNHcH3n+EnoAVfozGe3pPVdv8yY653eCM0Nx7L8/cluqUBwn0P39tFV6A+LYXHuRfD1yEdjaaL03248vr2/mDM9fKsg1XSfVDwg0KUSLvr4Xu76MpGeLzrHn5zs6pT0R6+RtOFCQhQFcn7fjBm1GTuSojtBdFeKkhyFiNy8c3w8MG9BmoNVSCbtYev0XR5ep+uRLBsn3vXkVI1ZPPFcI6NmAz3++lUHYaAIIhINMYmR3GngVpLa2S9aKvRdG36nKG27oZ1KnJVnnuGD4+LjM1Qs23dGaVZeVgNLvd2f6aIONc9/BrreZFJxtnjYbTgazSeIKGfCk+4m55pG4zuixk6NmIyVB97d6qLq7xcZWujdQM1Z6kpsl7Dy59K3EALvkbjCYRQYZ19S93rj5+7CoIjIGWocbYZjRF98b1ddGUjIlE1UDta6fy5NsGP1IKv0Wh6T4baYtUf31XyVkHaGAg0cnSFwRiRi+/ttgo2bJ05q4ucP1cLvkajaaHP6Wq75zvXzm+ohSObfTucA62qbV1cuG2sU/nsZnj4UclqW13g/Lm1JaqNRkiksTZ5EC34Go2niEmHlGGw/TPXzs9fr2LjvpyhA0rwwuNd9/BtOfjdUo2zyVFaPHwXBL+mCCITjLXHw2jB12g8yZCLVB69K83F8vxgwdaGO7n4ldZZtqZ6+IXOn1tT5FcZOqAFX6PxLEMuVtttnzp/7t4fICHLP2LEsW70xS+11irEmdAJNDxeDTCp0YKv0WjcJaGvarWwdb5z59UUqwyfQed7xi6jictUg19cycUv26eKrmyLv94kIEB5+S6FdEr84824FVrwNRpPM+Ri1d7YmWrUbZ+CbIKhszxnl5EkZIGl3rWwTule1TrCrEykqGTnQzpSKg/fj3LwQQu+RuN5bJOvnAnrbJ2vRNSX8+9bk5iltsU5zp9butfcwS6RLgh+fYXqYNqVQjpCiMeEEIes82w3CCFmtnHcOUKInUKIHCHEg+7cU6PxO+L7QI+RsOVjx46vOqImZg2d5TezUknsr7bFu5w7T0oVw4/vY7xNjhKV4rzg1xSrbVcSfCvPSSlHWr8WnrhTCBEIvATMAAYDVwohfGwSs0bjYYZcDPnroGx/x8du+xSQMMRPwjmgWhSExULJbufOqy1VVa6mCn6yWrR1piLaD4uuwDshnXFAjpRyr5SyAZgHXOiF+2o0voMtrLP5w46P3fIxJA+G5IEeNclQhFBefrGTgl+6V23N9vCbLc51zay1efhdT/DvEEJsEkK8IYSw1yc0DWi9kpNnfc4uQog5Qog1Qog1RUUulDtrNL5IXCb0nQo/vdj+hKWKPMhd4V/evY3ELNcF34yUTBtR1rCMM6mZLR5+JwvpCCG+EUJssfN1IfAK0BcYCRwGnrF3CTvPtdmaTko5V0qZLaXMTkryrx+mRtMu059U4YulT7d9zNZP1NZfsnNak5ilWjnXO9GIrHQvICCul8fM6hBXqm1tMXw/y9LpMA9KSjnNkQsJIf4NfGFnVx7QOsE2Hch3yDqNpjORMgRGXQ2r5sLYm1SOfmtqimHFy5A66uR9/kCCNVOnZLdq+OYIpXtV/n1QqOfs6ogWwXfSww+LUSMe/Qh3s3Rat7e7GNhi57DVQJYQorcQIgS4AnCxuYhG4+dMeRgCQ+CbR49/vrkJPrpJif75z5tjm7u4kppZts/clExwrYFaTbHfeffgfgz/b0KIzUKITcAU4F4AIUSqEGIhgJTSAtwBfAVsBz6QUm51874ajX8SnQKn3QvbP4edXx4bvPHdn2Dv93DuM9BjhKkmukxcb9WmwJnUzNK95i7YAoR2g8BQ5z18P4vfgwMhnfaQUl7TxvP5wMxW3y8ETkrZ1Gi6JKfeDuvfgfeuUCKZORHW/xdGXwuj7f5L+QdBIWpx2tHUzLpy1WLYbA9fCOdz8WuK/TLspittNRpvExIBc36A819Q3u3Geaowa8bfzbbMfZxJzbQNeDfbwwfn++l0RQ9fo9G4SEQ8jLlOfdVXqLh+cJjZVrlPYj/Ys0StSQQEtn9sqS8JfopjRXGgXlttiV8KvvbwNRqzCYuB4HCzrTCGxP7QdNSxRnEtOfiZHjXJIaKSHM/DrysDpN8VXYEWfI1GYyQtqZkOZOqU7oOo7r4xIjAqRcXlmywdH+unbRVAC75GozGSltRMBzJ1fCFDx0ZUMiCPtUxoDz+tsgUt+BqNxkgiEiA8zrGF2zKTu2S2xplqWy34Go1Gg0pxTMjqOKTTUKOGl5udkmkj0lZ85UD/rpoSte2ChVcajUZzPIn9oWhn+8fYMmJ8xsN3otq2pggQKtPKz9CCr9FojKXHCJXxYku7tEfhdrX1leIlZwU/IqHjtFMfRAu+RqMxlr5T1Xbvd20fs/d7lY7qKyMcQyIhJPpYfL49/LToCrTgazQao0noqzpg7llif7+UsOc76HOGb3nJUUkOevjFfpmSCVrwNRqN0QgBfafA3qX289qLd0NlHvSZ4n3b2sPRfjq1WvA1Go3mGH2mwNEKyF9/8j6b59/X1wTfwX46OqSj0Wg0rehzBiDsh3X2LIH4vr7RUqE1jnj4lgbV+0gLvkaj0ViJiIfUkScv3FqOwv5lxxZ2fYnIZKgvVza2ha0SNyLBKyYZjRZ8jUbjGfpOhdxVx8+4zV0FjbW+KfjdUtW2Iq/tY2whH1sap5+hBV+j0XiGPlNANsH+5cee27MEAoIg8zTz7GqLxP5q217RmG18Y7yP1A84iRZ8jUbjGTLGQXDk8XH8PUsgfRyEdTPPrrZIsgp+cXuCvxNEgO8UjDmJWwNQhBDvAwOs38YC5VLKkXaO2w9UAU2ARUqZ7c59NRqNHxAUqjz5XV/BgBkq7n14I0z5vdmW2ScsRrVrLmqn02fxLrXYHBTqNbOMxN2ZtpfbHgshngEq2jl8ipTSgd6jGo2m0zDofNj9Ffx31rHnfDF+byOpf/seftEuSBzQ9n4fx5ARh0IIAcwGfPg3qdFovM7oa6DfNNX7vmwfNDVC2hizrWqbxAFqxrCUqoCsNU0W1QW0/3RzbDMAo2baTgIKpJRtNcGWwGIhhAT+JaWc29aFhBBzgDkAPXv2NMg8jUZjGt16qK/MiWZb0jFJA6ChSrVutmXt2Cg/AM2NxxZ3/ZAOBV8I8Q3Q3c6uh6SUn1ofXwm8185lJkop84UQycDXQogdUsql9g60vhnMBcjOzpYd2afRaDSG0TpT50TBt2XvdOaQjpRyWnv7hRBBwCygzc9pUsp867ZQCDEfGAfYFXyNRqMxjSSrmBfvOrn1gy22bxvj6IcYkZY5DdghpbRbrSCEiBRCRNseA9OBLQbcV6PRaIwlKgVCY+zn4hfvVvvDY71ullEYIfhXcEI4RwiRKoRYaP02BVguhNgIrAIWSCkXGXBfjUajMRYhrJk6dlIzi3b6dfweDFi0lVJeb+e5fGCm9fFeYIS799FoNBqvkDgAdi8+/jkp1ZvA8Nnm2GQQutJWo9FoWpPUX41orCs79lx1ARyt9HsPXwu+RqPRtMaWhVPcKsu8JUNHC75Go9F0HpLsNFGzxfST/DclE7TgazQazfHE9oLA0ONbLBTvUkPOo3uYZ5cBaMHXaDSa1gQEqlz71k3Uinaq505st+BnaMHXaDSaE0nsf7KH7+fhHNCCr9FoNCeTNADKDsCRzVBTonrr+PmCLRjXPE2j0Wg6D6mjAAmvngZYwzha8DUajaYT0v9suO1nKNwGRTugphj6nG62VW6jBV+j0WjskTJYfXUidAxfo9Fougha8DUajaaLoAVfo9Fougha8DUajaaLoAVfo9Fougha8DUajaaLoAVfo9Fougha8DUajaaLIKSUZtvQJkKIIuCAi6cnAsUGmuMP6Nfc+elqrxf0a3aWXlLKJHs7fFrw3UEIsUZKmW22Hd5Ev+bOT1d7vaBfs5HokI5Go9F0EbTgazQaTRehMwv+XLMNMAH9mjs/Xe31gn7NhtFpY/gajUajOZ7O7OFrNBqNphVa8DUajaaL0OkEXwhxjhBipxAiRwjxoNn2eAMhxBtCiEIhxBazbfEGQogMIcR3QojtQoitQoi7zbbJ0wghwoQQq4QQG62v+XGzbfIWQohAIcR6IcQXZtviDYQQ+4UQm4UQG4QQawy9dmeK4QshAoFdwFlAHrAauFJKuc1UwzyMEGIyUA28I6UcarY9nkYI0QPoIaVcJ4SIBtYCF3Xm37MQQgCRUspqIUQwsBy4W0q5wmTTPI4Q4tdANtBNSnme2fZ4GiHEfiBbSml4sVln8/DHATlSyr1SygZgHnChyTZ5HCnlUqDUbDu8hZTysJRynfVxFbAdSDPXKs8iFdXWb4OtX53HW2sDIUQ6cC7wmtm2dAY6m+CnAbmtvs+jkwtBV0cIkQmMAlaabIrHsYY2NgCFwNdSyk7/moF/APcDzSbb4U0ksFgIsVYIMcfIC3c2wRd2nuv0XlBXRQgRBXwE3COlrDTbHk8jpWySUo4E0oFxQohOHb4TQpwHFEop15pti5eZKKUcDcwAbreGbA2hswl+HpDR6vt0IN8kWzQexBrH/gh4V0r5sdn2eBMpZTnwPXCOuZZ4nInABdaY9jxgqhDiv+aa5HmklPnWbSEwHxWqNoTOJvirgSwhRG8hRAhwBfCZyTZpDMa6gPk6sF1K+azZ9ngDIUSSECLW+jgcmAbsMNUoDyOl/J2UMl1KmYn6X14ipbzaZLM8ihAi0pqIgBAiEpgOGJZ916kEX0ppAe4AvkIt5H0gpdxqrlWeRwjxHvAzMEAIkSeEuMlsmzzMROAalMe3wfo102yjPEwP4DshxCaUY/O1lLJLpCl2MVKA5UKIjcAqYIGUcpFRF+9UaZkajUajaZtO5eFrNBqNpm204Gs0Gk0XQQu+RqPRdBG04Gs0Gk0XQQu+RqPRdBG04Gs0Gk0XQQu+RqPRdBH+H2CPrSMV0KsfAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import torch\n",
    "from matplotlib import pyplot as plt\n",
    "\n",
    "x = torch.linspace(0, 5, 100,\n",
    "          requires_grad=True)\n",
    "y = (x**2).cos()\n",
    "s = y.sum()\n",
    "[dydx] = torch.autograd.grad(s, [x])\n",
    "\n",
    "plt.plot(x.detach(), y.detach(), label='y')\n",
    "plt.plot(x.detach(), dydx, label='dy/dx')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(Note that in the example above, because the components of the vector space are independent of each other, we happen to have `dy[j] / dx[i] == 0` when `j != i`, so that `d(y.sum())/dx[i] = dy[i]/dx[i]`.  That means computing a single gradient vector of the sum `s` is equiavlent to computing elementwise derivatives `dy/dx`.)\n",
    "\n",
    "**Detaching tensors from the computation history.** Every tensor that depends on `x` will be `requires_grad=True` and connected to the complete computation history. But if you were to convert a tensor to a regular python number, pytorch would not be able to see the calculations and would not be able to compute gradients on it.\n",
    "\n",
    "To avoid programming mistakes where some computation invisibly goes through a non-pytorch number that cannot be tracked, pytorch disables requires-grad tensors from being converted to untrackable numbers.  You need to explicitly call `x.detach()` or `y.detach()` first, to explicitly say that you want an untracked reference, before plotting the data or using it as non-pytorch numbers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise\n",
    "\n",
    "Plot the polynomial y=x<sup>3</sup>-6x<sup>2</sup>+8x and its derivative, instead of cos(x<sup>2</sup>)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO: set y to the given polynomial of x\n",
    "y = 'TODO'\n",
    "\n",
    "# TODO: use autograd to compute the derivative\n",
    "\n",
    "# TODO: plot the results.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Backprop and In-place gradients\n",
    "-------------------------------\n",
    "\n",
    "In a typical neural network we will not just be getting gradients with regard to one input like `x` above, but with regard to a list of dozens or hundreds of tensor parameters that have all been marked with `requires_grad=True`.  It can be inconvenient to keep track of which gradient outputs go with which original tensor input.  But since the gradients have exactly the same shape as the inputs, it is natural to store computed gradients in-place on the tensors themselves.\n",
    "\n",
    "**Using `backward()` to add `.grad` attributes.** To simplify this common operation, pytorch provides the `y.backward()` method, which computes the gradients of y with respect to every tracked dependency, and stores the results in the field `x.grad` for every original input vector `x` that was marked as `requires_grad=True`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([-0.0000e+00, -2.5765e-04, -2.0612e-03, -6.9560e-03, -1.6485e-02,\n",
      "        -3.2185e-02, -5.5575e-02, -8.8145e-02, -1.3133e-01, -1.8650e-01,\n",
      "        -2.5487e-01, -3.3752e-01, -4.3528e-01, -5.4869e-01, -6.7791e-01,\n",
      "        -8.2262e-01, -9.8193e-01, -1.1543e+00, -1.3373e+00, -1.5279e+00,\n",
      "        -1.7218e+00, -1.9138e+00, -2.0978e+00, -2.2665e+00, -2.4118e+00,\n",
      "        -2.5246e+00, -2.5954e+00, -2.6144e+00, -2.5720e+00, -2.4592e+00,\n",
      "        -2.2684e+00, -1.9940e+00, -1.6330e+00, -1.1861e+00, -6.5843e-01,\n",
      "        -5.9786e-02,  5.9438e-01,  1.2829e+00,  1.9791e+00,  2.6508e+00,\n",
      "         3.2620e+00,  3.7737e+00,  4.1467e+00,  4.3434e+00,  4.3315e+00,\n",
      "         4.0872e+00,  3.5983e+00,  2.8676e+00,  1.9159e+00,  7.8273e-01,\n",
      "        -4.7262e-01, -1.7729e+00, -3.0265e+00, -4.1327e+00, -4.9894e+00,\n",
      "        -5.5028e+00, -5.5970e+00, -5.2252e+00, -4.3782e+00, -3.0925e+00,\n",
      "        -1.4526e+00,  4.1007e-01,  2.3249e+00,  4.0956e+00,  5.5192e+00,\n",
      "         6.4094e+00,  6.6222e+00,  6.0798e+00,  4.7897e+00,  2.8560e+00,\n",
      "         4.7794e-01, -2.0646e+00, -4.4405e+00, -6.3087e+00, -7.3680e+00,\n",
      "        -7.4080e+00, -6.3531e+00, -4.2917e+00, -1.4813e+00,  1.6739e+00,\n",
      "         4.6748e+00,  7.0040e+00,  8.2157e+00,  8.0255e+00,  6.3823e+00,\n",
      "         3.5034e+00, -1.3781e-01, -3.8789e+00, -6.9824e+00, -8.7814e+00,\n",
      "        -8.8286e+00, -7.0156e+00, -3.6318e+00,  6.6059e-01,  4.9416e+00,\n",
      "         8.2239e+00,  9.6828e+00,  8.8724e+00,  5.8738e+00,  1.3235e+00])\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABJbklEQVR4nO2dd3iUVdqH75NeSU8gBUIJvRMQQVAQUbCjoq69LOra1111V9ey6lbL6tqWte+6on6KDURUVECl906oCYH0XkgmOd8fZyYEmCRT3pl3Jjn3deV6J/O2Z1J+88xzniKklGg0Go2m8xNgtgEajUaj8Q5a8DUajaaLoAVfo9Fougha8DUajaaLoAVfo9FoughBZhvQHomJiTIzM9NsMzQajcZvWLt2bbGUMsnePp8W/MzMTNasWWO2GRqNRuM3CCEOtLVPh3Q0Go2mi6AFX6PRaLoIDgu+EOINIUShEGJLq+fihRBfCyF2W7dxbZx7jhBipxAiRwjxoBGGazQajcY5nInhvwW8CLzT6rkHgW+llH+xCvmDwAOtTxJCBAIvAWcBecBqIcRnUsptrhjc2NhIXl4e9fX1rpzut4SFhZGenk5wcLDZpmg0Gj/FYcGXUi4VQmSe8PSFwBnWx28D33OC4APjgBwp5V4AIcQ863kuCX5eXh7R0dFkZmYihHDlEn6HlJKSkhLy8vLo3bu32eZoNBo/xd0YfoqU8jCAdZts55g0ILfV93nW5+wihJgjhFgjhFhTVFR00v76+noSEhK6jNgDCCFISEjocp9qNBqNsXhj0daeMrfZolNKOVdKmS2lzE5KsptK2qXE3kZXfM0ajcZY3BX8AiFEDwDrttDOMXlARqvv04F8N++r0Wg0/sf+5ZC/wbTbuyv4nwHXWR9fB3xq55jVQJYQorcQIgS4wnqeRqPRdC3m3wqf/Mq02zuTlvke8DMwQAiRJ4S4CfgLcJYQYjcqC+cv1mNThRALAaSUFuAO4CtgO/CBlHKrsS9Do9FofJyaYqjIhcKtUGCOBDqTpXNlG7vOtHNsPjCz1fcLgYVOW+eD/OEPfyAxMZG7774bgIceeoiUlBTuuusuky3TaDQ+zeENxx5v+gDOetzrJvh0L52OePzzrWzLrzT0moNTu/Ho+UPa3H/TTTcxa9Ys7r77bpqbm5k3bx6rVq0y1AaNRtMJscXuM8bD5v+DMx+FAO82O9CtFZwkMzOThIQE1q9fz+LFixk1ahQJCQlmm6XRaHydwxshvg+MvQkq8+Dgz143wa89/PY8cU9y880389Zbb3HkyBFuvPFGU2zQaDR+xuENkDYGBsyE4AjY/AFkTvSqCdrDd4GLL76YRYsWsXr1as4++2yzzdFoNL5ObSmUH4QeIyE0CgaeC1s/AUuDV83Qgu8CISEhTJkyhdmzZxMYGGi2ORqNxtc5vFFte4xQ22Gzob4ccr72qhla8F2gubmZFStWcNNNN5ltikaj8QdOFPy+UyAiAbZ85FUztOA7ybZt2+jXrx9nnnkmWVlZZpuj0Wj8gcMbILYnRMSr7wODIeMUKNrpVTP8etHWDAYPHszevXvNNkOj0fgThzeq+H1rolIg17sp3drD12g0Gk9SXwGleyF15PHPR6VAbTE0NXrNFC34Go1G40kOb1JbW/zeRnSK2tac3AbeU2jB12g0Gk9ia6lgL6QDUHXEa6ZowddoNBpPcngjdEuHyMTjn4/qrrbV9rrKewYt+G7y2GOP8fTTT7d7zHvvvcdTTz110vOZmZkUFxd7yjSNpmuw+GH48kE4WmW2JfY5vPHkcA4cC+lUaw+/U7Fo0SLOOeccs83QaDofDTWw4hVY+Qq8PAH2LDHbopOpOATxdmZRR1onwlYVeM0ULfgu8NRTTzFgwACmTZvGzp07aWpqYvTo0S37d+/ezZgxYwA1gHzDhg2MHj2akpISpk+fzqhRo7jllluQUk16XL16NcOHD6e+vp6amhqGDBnCli1bTHltGo1fkbcGmi1wxu8gOAz+czH8+ILZVh2joQYaayDSzrjWoBAIj4dq7wm+f+fhf/kgHNls7DW7D4MZf2lz99q1a5k3bx7r16/HYrEwevRoxowZQ0xMDBs2bGDkyJG8+eabXH/99QCsX7+eESNGIITg8ccf57TTTuORRx5hwYIFzJ07F4CxY8dywQUX8PDDD1NXV8fVV1/N0KFDjX1dGk1n5ODPgIBTboWJ98A7F8CG/8FEH5lPYcvAsSf4oBZuvSj42sN3kmXLlnHxxRcTERFBt27duOCCCwDVQfPNN9+kqamJ999/n1/84heACufMmDEDgKVLl3L11VcDcO655xIXF9dy3UceeYSvv/6aNWvWcP/993v5VWk0fsqBnyBlKITHKg+/10Qo2e31pmRtUm0V/Khk+/ujvSv4bnv4QogBwPutnuoDPCKl/EerY85AzbvdZ33qYynlH929d3ueuCcRQpz03CWXXMLjjz/O1KlTGTNmTEuP/MWLF/PRRx+1ey5AaWkp1dXVNDY2Ul9fT2RkpGeM12g6C02NkLcaRl197LmUISrEU7JbPTabFg8/0f7+qO7qTctLuO3hSyl3SilHSilHAmOAWmC+nUOX2Y4zROxNYvLkycyfP5+6ujqqqqr4/PPPAQgLC+Pss8/mtttu44YbbgCgoqICi8XSIv6TJ0/m3XffBeDLL7+krKys5bpz5szhiSee4KqrruKBBx7w8qvSaPyQI5ugsRZ6nnrsueTBaluwzRybTqTGmnLZZkgnWWXpWNfzPI3RIZ0zgT1SygMGX9dnGD16NJdffjkjR47kkksuYdKkSS37rrrqKoQQTJ8+HYCvv/6aadOmtex/9NFHWbp0KaNHj2bx4sX07NkTgHfeeYegoCB+8Ytf8OCDD7J69WqWLPHBbAONxpc4YJ0Y1WvCsecS+kFAkBoU7gt0FMOP7g5NDapVshcwetH2CuC9NvadKoTYCOQDv5FS+shvxHkeeughHnrooZOeX758OTfeeGNLj/xFixZx8803t+xPSEhg8eLFLd8/99xzAFx77bVce+21AAQGBrJy5UpPmq/RdA4O/gxxvZVo2ggKgcT+vuPhVxdBaAwEhdrf31JtWwDhcfaPMRDDBF8IEQJcAPzOzu51QC8pZbUQYibwCWC3t7AQYg4wB2jxgP2Biy++mD179hznmb/22msmWqTRdGKkVILf3059S/JgyPURp6mmCKLa8O7hmOBXF0DyQI+bY2RIZwawTkp50pKzlLJSSlltfbwQCBZC2F3FkFLOlVJmSymzk5La+UH5GPPnz2fTpk0kJraxOKPRaIyjeBfUlkDP8SfvSxkMFbmqS6XZ1BS1Hc6BY59OvJSpY6TgX0kb4RwhRHdhTU8RQoyz3rfE1RtJLy1w+BJd8TVrNG1iy2zpOeHkfcnW7JzCHd6zpy06EnxbuqaXGqgZIvhCiAjgLODjVs/dKoS41frtpcAWawz/BeAK6aKChYWFUVJS0qUEUEpJSUkJYWFhZpui0fgGB1coIU3oe/K+FGumji8s3HYk+KHdICjcax6+ITF8KWUtkHDCc6+2evwi8KIR90pPTycvL4+iIu/1kPYFwsLCSE9PN9sMjcY3OPizSse0V9cSkwEh0eYv3DZZoLa07aIrUPZHJfuX4HuT4OBgeve204hIo9F0DSxHofwAjLzK/n4hIHkQFJos+LUlgGy76MpGdHe/jOFrNBqN56nIU9vYjLaPSRkMBVu9VtBkl5aiq3Y8fFCZOl7qmKkFX6PR+Bfl1rrO2HbStpOHqGKmqsNeMckuHRVd2YhK8VpPfC34Gk1XpXAHvDsb9i012xLnKM9V2/YEv2Xh1sSwTrWDgh+dolJIG+s9bpIWfI2mqyElrPsPzD0Ddn8FC+5TC4z+QvlBEIEQndr2Mb7QU8fm4bdXeAXHF195GC34Gk1XorkJ5t8Kn90BGWPh3GdVEdPGtjqi+CDlB6FbGgS2k3MSEa86UZrp4dcUQWCISr1sDy/OttWCr9F0JfYvg03z4LR74ZpPIPtGSMuG7//slZCCIVTkth/OsZE8CIpMLL6qKVILtm20RG/BlrbphTi+FnyNpiux7TMIjoDJ90NAoBKjaY9B5SFY7Se9n8oPtp+hYyMmzWsVrHapKeo4JROOtVfwgq1a8DWarkJzM+z4ArLOgpCIY8/3ngR9p8KyZ3yj/0x7WBqgMt8xDz/Kmt/e3OR5u+xRXdh+0ZWNyCQQATqko9FoDCRvlRLAQRecvO/MR6CuFNa84X27nKHyECAdE/zo7iCboabY42bZpaa44wwdUJ+0IhJ1SEej0RjIts/UImLW9JP3pY5Suev7lnnfLmcoP6i2MQ6EdFpCJSbk4kupCq8cEXywzrbVHr5GozECKWH75yp0E9ZG1kjGWMhbo0I/vopN8B3y8HuorRlx/KOVapKVo4IflaJj+BqNxiDy10PFQRh0ftvHpI+DoxUqTdNXqchV8e5uaR0f29Jr3gTBtxVdORLDBxXSqSv1nD1WtOBrNF2B7Z+pYqUBM9s+JmOc2uat8o5NrlB+UHnuQSEdHxvp3V7zx9HSVsHBgUhhMV5ZMNeCr9F0dqRU8fvek1RBUlsk9FNzVX1lPKA9yg86Fs4B9aYQkWiS4NsapzkY0gmLgfpKj4fTtOBrNJ2d4t1Quqf9cA6onPz0sZC72jt2uUK5g0VXNqK7m+zhOxjSCYsBpIr9exAt+BpNZyfPKuCZkzs+NmMcFO+EujLP2uQKTRaVlulIho6N6O7mZOnUFAMCIhI6PBSA8Fi19XBYRwu+RtPZObwBQqJUyKYj0m1x/LUeNcklKg+BbHLew/fScJHjqC5U4bP2+v20JixGbf1B8IUQ+4UQm4UQG4QQa+zsF0KIF4QQOUKITUKI0UbcV6PROED+eugxAgIc+HdPG6OyYHxx4bbCgbbIJ2JWtW1Hs2xPxJ8E38oUKeVIKWW2nX0zgCzr1xzgFQPvq9Fo2qKpEY5sVoVVjhAapQqwcn1Q8J3JwbdhVrVtFxD89rgQeEcqVgCxQogeXrq3RtN1KdoBlnrHBR9aFWCZ1IOmLVqqbNMdP8esattOLvgSWCyEWCuEmGNnfxqQ2+r7POtzJyGEmCOEWCOEWFNUVGSQeRpNFyV/vdo6I/jp46ChytzWwvYoz7Xm4Ic6fo5Z1bbVzgp+rNrWl3vCmhaMEvyJUsrRqNDN7UKIE9MB7DWEtjtdWEo5V0qZLaXMTkpy4gem0WhOJn+DGsAR19vxc2wFWL4W1ik/4FyGDphTbdtkURXLjmbogHVIivAPD19KmW/dFgLzgXEnHJIHtP5NpQP5Rtxbo9G0gzMLtjbi+0B4PBzysUwdZ4qubJhRbWsTbVuqpSMEBCjR93XBF0JECiGibY+B6cCWEw77DLjWmq0zHqiQUpo4Tl6j6QJYGqBgi3PhHFAFWEkDoWSPZ+xyheZmlZbpyOCT1phRbWsLy9jCNI7ihfYKDiaJtksKMF+oMV5BwP+klIuEELcCSClfBRYCM4EcoBa4wYD7ajSa9ijcpjo2Oiv4AAl9YddXxtvkKrXF0Gxpf3B5W3i72rauXG3D45w7zx8EX0q5Fxhh5/lXWz2WwO3u3kuj0TjB4Q1qmzrS+XMT+kHNf5QA2TJIzMRWPBWd4vy53q62tVUpOxPSsR1ve7PwEEZ4+BqNb5PzDRzeCMGREByuiou6DzXbKs+Tv16JtTMLtjZsVbkleyDNB+okq6yCH9Xd+XOju0PBVmPtaQ93Qjql+4y25ji04Gs6L3Xl8OUDsGne8c8HBMH5z8Ooq00xy2vkr1fhHGEvSa4DWgQ/xzcE35Zl42h/+da0rrYNCDTWLnu46uH7Q0hHo/FJ9i2F+beq2O3pD8CEu1Q8u74CFvwaPr1ddZE881HnMlj8hcZ6KNgGE+5w7fz43oBQgu8L2EI6US6GdGzVtq6EhJylky/aajS+xZHN8O5lKmf75q9VCMdGRDz84kP48rfw4z+gMh9mzXXNC/ZlCrdCcyP0GOna+UGhKgXSVwS/qkClLYZEOH9u62pbbwh+XbkKHzoypKU1YTGq4K3J4njTNSfphK6NpktTXwEfXKu8qxsWHi/2NgKD4Nxnlee/+QPY+aXXzfQ4hdvVtvsw16+R0M93BL+6wDXvHrxfbVtX7nw4B459IvBgT3wt+JrOg5Twya9Ugc7st9uP9woBk3+r8s2/+p0KgXQminZAYCjEZbp+jcQstWgr7RbFexe3BN/L1bb15c6Hc6BVP51yA405Hi34ms7DTy/Aji/grCeg5/iOjw8Mhhl/g7L98NM/PW6eVynapQTbnUXKhH7QUG1OP/kTqS5wPRzj7Wpblz18zzdQ04Kv6RyU7oVvn4BBF8D42xw/r8/pMPhCWPaMas7VWSjaAUkD3LtGQl+19YWwTpUbHr63q23rypwvugIt+BqNw3z9KASGKI/d2QXY6U+q7eKHjbfLDBpqVVgr0V3Bb5WaaSZHq6GxxnXBB+9W27od0tGCr9G0zYGfYPtncNo90M2FMQuxPVX64rZP1CcFf6dkNyDd9/C7pat1ALMFv6XK1oWiKxverLZ1NaRjO8eD1bZa8DX+TXMzfPV71WPlVBdzzgGyb1Sj/da9Y5xtZlG0U23dFfyAABXWMbuJWksOvgtFVzaiUtScWU/T1Kg+jWgPX6PxAJs/VBWl0x51LUfbRrdUyDob1r+r/mn9maKdIAIhvq/710roa76HbwvFuNJWwUZEAtSWeD7jqKVxWqzz54ZEKadDC75GYwfLUfj2j6p9wLDZ7l9vzHVQUwi7Frl/LTMp2qGE2tnCH3skZKn+Lk0W96/lKu5U2dqISICmo9BQY4xNbdHSVsGFRVshPF5tqwVf479snAeVeXDmI8a0R+h3lirSWfu2+9cyk6KdkNjfmGsl9FMVu+UHjLmeK1QXQECwayJqwzZ9qrbEGJvawtW2Cja04Gs0dmhugh+fV60D+kwx5pqBQTDyKtVd019TNC0NauE5aaAx12vdNdMsqgpU/N6dN3VvCb47IR1QbxS68EqjOYHtn0HpHjjtXmP74Iy+BpCw4V3jrulNSveAbPKA4JsYx3enytZGZKLaag9fo/EzpITlzykxGnS+sdeOy1SfGNb9R2UA+RtFO9Q2yaCQTkS8Ei+zBd+dlEzwoofvRgwftOBrNCexZ4kaaDLxbs/0Nx9xpVob8LUh3o5QtAsQarHVCIRQQ83LPDuYo12qC9xLyQT1xgXeC+m4OiXM1wVfCJEhhPhOCLFdCLFVCHG3nWPOEEJUCCE2WL8ecfe+mi7M8udU3v3wyz1z/ayzVFrjLj/solm0A+J6uZeieiKxGeataTRZVB97d1IyAUJj1O/UGyGdkGjX2xv7uuADFuA+KeUgYDxwuxBisJ3jlkkpR1q//mjAfTVdkfwNsH8ZnPor1bPdE0TEQ89T/bNtctFO91sqnEhMBlTkmdM1s6YIkO57+AEBKqxTU2yIWW3iapWtjfBYaKxVi+8ewG3Bl1IellKusz6uArYDae5eV6Oxy6q5arjE6Gs9e58BM6Bwm+qk6S80WVSs3d0K2xOJ7QmWOs+LpT1sLY3djeHDseIrT1JX5vqCLRw710NevqExfCFEJjAKWGln96lCiI1CiC+FEEPaucYcIcQaIcSaoqIiI83T+DvVRaqyduSVrsdIHWXADLXd6UdFWOUHVHGRURk6NmIy1LbioLHXdYQqA4qubEQkQG2p+9dpj/py9zx8D7dXMEzwhRBRwEfAPVLKE0e2rAN6SSlHAP8EPmnrOlLKuVLKbClldlJSklHmaToD695Sc2nHzfH8vRL6quIlf4rjG9VD50RirYJvRhzfiCpbGxHx3lm07eyCL4QIRon9u1LKj0/cL6WslFJWWx8vBIKFEIlG3FvTRWhqhNWvQ9+pxgtaW/Q/B/b/6PHB0oZRvEttEw3K0LHR4uGbKfhuxvBB5eJ7Y9HWrZCOZ6deGZGlI4DXge1SymfbOKa79TiEEOOs9/XwT17Tqdj+uWpvO+4W791zwEzVViDnW+/d0x1KctR0J6PDXeGxaoC4WR5+eJwxC/QRCVBX6tn6iroyNz1867kecjKMGI0+EbgG2CyE2GB97vdATwAp5avApcBtQggLUAdcIaUvDMrU+A0r/wVxvSFruvfumTEOwuNVM7Whs7x3X1cpyTlWGWs0MRnmePhVR9xPybQRkQCyWXnPtrx8I2msB0u9T3v4bgu+lHI50G5tu5TyReBFd++l6aIc3gi5K+DsPxnTJM1RAgKh/9kqPbPJ4nputbcoyTm22Gw0ZuXiVxcaE86B46ttPSH4NpF2p8mbP8TwNRqPsvo1CApXjc28TdZ09Y+cv97793aGunKVs+5RD9+ELJ3qI8akZILn2yu42zgNIDhcdQbVgq/pktSVw6YPYfhl7v0juUrvyWq7f6n37+0Mtm6WRrVUOJHYDCVC9Scm4HkQKT3j4XuqnsDdxmng8Z74WvA1vs3G91TRT/ZN5tw/MhGSB8O+Zebc31Fszc086eGDd+P49RUqJm5ESiZ4wcO3NU6Lde864bFa8DVdEClVKmZaNqSONM+OzEmQu9Jj5e6GULJb9YqJy/TM9WN7qq034/i2GbRGLtqCF0I6bsTwQXn4HhpkrgVf47vsW6qEbOzN5trRe5Lqb+LL3TNLclTTNCPGGtrDDA+/xib4BhVghkRAcITnBN+IkA7okI6mi7L6NeUtDbnYXDt6TQSEatrmqxTneC5+DxCZBIEhUO7FhVubhx9pUAwfPNtPx93WyDa04Gu6HJX5sGMBjLoGgsPMtSUiHroPVZ84fJHmZjXpylPxe1DpsDHp3vXwW0I6BsXwwbPtFerKVBtmd2c0aMHXdDnWvKmKZLJvNNsSReZkyF2limt8jap8FXJK9KDggwrreDOGX1Oo1iXcjYm3xpMefn05hBtQ5Tz+V3D5f92/jh204Gt8D0sDrH1LDSKJ7222NYrek1QnyrzVZltyMp7O0LER6+Vq2+pCFUoystguwoP9dOrKjXlzShoAPU9x/zp28PHSQdeY/erPHLU0OXawEC1lwkKokuEAIQgQAiHU48AAQUCAIMj6FRwYQFCgICQwgJCgAEKDAgkLDiA8OJDwkEAiQoKICgsiKjSQmPBgYsKD6RYeTFxECMGB+j22Q7Z/pry7sb802xIAGpua2RY4mGEEsPiLD/koJpwGSzPNUtIvOYqhqTEMT4+hX3IUwsiB6o5SvFttPRjDL69t4HBdLIOqC5jzxo+kJsaSmRDB2N7xDEn1UKtqI3PwgdoGC0X1YaRUFHLH22s4Z2h3zhnanahQg2TQ3cZpQHOz5L8rD7D+YDnPzh5h+N9TpxT82IhgGpo6jqNJCbLlsWx5rlnKlq2luZmjFkmThKbmZixNksamZizNkgZLc8tXvaWJxqaO2wPFRgSTEBlCSrcwuncLo3tMGGlx4WTERdAzPoK0uHD9prD6NZVe2G+aqWZszqvg7Z/3s2DTYeoam/g0JJPupavJbbqU0KAAmiXMW5VLXeN+AAZ2j+a6CZlcODKViBAv/muV7IGQKOMqUltxuKKOP3yylSU7CrhYWHgmBJrKc/lwfxU1DcqpmjUqjQdmDCSlm8FrLTXGCH5Ts+Svi3bw1k/7mSPr+E1wLTn5JfxmewEPf7KZ84en8scLhxIe4mbsva4Mkge5fHpOYTW/+3gTq/eXMSkrkbrGJsP/jjql4M+9NtuU+zY2NVPX2ETNUQs1Ry1U1VuorLdQUddIeW0DpTUNlFQ3UFx9lILKelbsLaGw6iiW5mNvFEEBgl4JEfRLjqJ/SjSDe3RjcGo3MuIiCAgwwXv0Nke2wMGf4awnvNs3pxXf7Szkn9/uZt3BciJCArlwZCqT+yfR78AMItfNZdGvsltmxjY1S/YWVbNiXyn/W3mQ3328mT8v3M5dZ2Zx/YRMgrzx5l2yW/XvN9AblFIyf/0hHv1sK5Ymya2n92VWfAAsfJXXL0xG9plCUdVR3vppP68t28eirUe4d1p/bp7U2zivtLpIFb25QX1jE3fPW89XWwuYNTqNi6OGw6oP+e72YawrC+PjdYd4b9VBDpTW8sb1Y93z9uvKXfbwP1idy8OfbiE8OJCnLxvBJaPTPPJpsVMKvlkEBwYQHBhAt7Bgh89papYUVNaTW1rLwdJa9hXXkFNYze7Car7eVoDtvaBbWBAje8YxMiOWMb3iyO4VR6RRH0V9idX/hqAwGHW112+9u6CKJxds54ddRfSMj+DR8wdzyZj0Y7/P0DNgzUsqjt/ndAACAwRZKdFkpURz9Sk9WXOgjJe+y+HJBdv5ZMMh/nzxcIale3g6V0kOpI0x7HL1jU3c98FGFmw+zNjMOP5+6QgyEyOhzOrBl+cihCC5Wxj3nzOQK8b25I9fbOOphdvJr6jjkfMGuy9WUioPP9L1HPzSmgZufns163PL+cN5g7nptN6w7QCsAlFbypheQxnTK55T+iRw7/sbuPb1lbx14zin/n+Ps7e+3KUY/toDpfx+/mbG90ng2ctHkBztuay0TqgY/kVggCA1NpzU2HBO6ZNw3L76xiZ2FVSxNb+STXkVrD9YxotLdtMs1SeB4ekxnNo3gdP7JzO6Z6x3vElPUl8Bmz6AoZd6ppthW7dtbOK5r3fx2vJ9RIQE8vC5g7j21ExCgk74eWaMVdvclS2C3xohBGMz43nz+rF8ueUIj362lQtfWs5dZ2Zx19Qsz3xCsxxVufHDrzDkcrUNFua8s5blOcU8cM5A5kzuQ6DN7m5pIAJOWrjtmRDBv68dwxNfbOeNH/dRe7SJP80aduw8V6gvV9PNXAzp1Dc2cd0bq9hZUMVLvxjNzGE91I6Wattj/XQuGJFKSGAAd763jqv+vZIPbjnV+fBOY52y18m2CqU1Ddzxv/Wkxobz8tWjXXuzcQIt+D5MWHAgw9NjGZ4ey5Xj1HPVRy2sP1jGz3tKWLG3hFd/2MtL3+0hOiyIyf2TOHtId6YMSCLaw384HmHdf1R64TjvVdZuyivn1x9sJKewmsuzM7j/nAEkRLUxbCM8DpIGKcFvByEEM4f1YGK/RB77bCv/+GY3W/MreXb2CON/L6X7VPqqARk61Uct3PjmatYcKOXpy0Zw6Zj04w8IDIboHnZTM4UQ/OG8QUSFBvLCkhzqLU08N3uk629ybubgP/rpVjYfquDf12Zz1uBW12ijvcI5Q7vz8lVj+OU7a3hywTaeuniYczd0ocq2uVly7/sbKKlu4ONfTfC42IMWfL8jKjSISVlJTMpSH3Ur6hr5MaeY73cWsmRHEQs2HSYkMICJ/RK4YGQqZw02MAvBkzRZ1JCTnhMgdZTnb9cseXFJDi8s2U1SVChv3ziO0/s7ED7oeQpsma+KnTpYY4gJD+bZ2SMYlhbDUwu3c/HLP/HatdkqPGIUJdYMHTdz8GsbLFz7+ko25lXw/BWjOH9Eqv0DYzKgIs/uLiEEv54+gNDgQP7+1U56xkdw33QXx1G2VNk6H9J5f/VB3l+Tyx1T+h0v9tBK8E8eZn7W4BTmTO7D3KV7OWNA8snntocLjdP+tXQvP+wq4smLhjI0zcNhPyt+oASa9ogJD2bmsB7MHNaDpmbJ+oNlfLX1CAs3H+He9zcSFryZswZ359Ix6ZzWL9G9j9meZOcC1W/97Kc8fquCynrumbeBn/eWcNHIVB6/cCgx4Q56VxnjVY1A0XZIGdLh4UIIbjytNwN7RHP7u+u45JWfePvGccb9g9tSMuP7unwJS1Mzt7+7jg255bx81WjOGdqj7YNj0jrsKfSrM/pysKSWfy7JISslmgvaevNoj5Y+Os6FdDbnVfCHT7cyKSuRe8/qf/IB4dZQYRu5+PdN78/y3cU88NEmRqRPItnRzKOWtgqxDh1+oKSG577Zxcxh3bnqlJ6O3cMAjBpifo4QYqcQIkcI8aCd/UII8YJ1/yYhxGgj7qs5nsAAQXZmPA+dO5hl90/h/249lcvGZLBsdxHXvbGK0/66hGcW7ySvrNZsU0/m55chthcMPNejt1m6q4iZzy9jQ245T182gn9cMcpxsQc19hA6DOucyIS+iXx02wTCggO5cu4KVu41qPinJEeFWcK6uXS6lJKH5m/hu51FPHHR0PbFHlQcvzK/3bmwQgieuGgoYzPj+O2HG9mUV+68YS6EdGqOWrj9f+tIjAzh+StG2XduAoOUKLch+KFBgbxw5UhqGyzc9+FGmps7TrUGWk27iu3wUCklj322lZDAAB49f4hXazeMGGIeCLwEzAAGA1cKIU7MpZoBZFm/5gCvuHtfTfsEWMX/iYuGsvL3Z/LyVaMZ0D2al77LYfLfvuOmt1bz3Y5Cx/+gPcmhtWqE4Sm3ut+HpA2amiXPfr2L695cRWJUKJ/fOfHkGLUjxPdRYYaDzgk+QJ+kKD689VSSu4Vy7Rur+HZ7gfP3P5HiXW7F75//djfvr8nlzqn9uOqUXh2fEJOuFidr2x8iEhIUwCtXjyExKpQ576yluPqoc4ZVF0JAkFMx8ScXbCO3rJbnrxxFfGQ7XUMjEtodgtIvOVo5TbuLeX+Ng5XFTrRG/npbAd/tLOKeaVnG1y50gBEe/jggR0q5V0rZAMwDLjzhmAuBd6RiBRArhOjAlXCDynyoOHTsqzJfDUOuLoSaEvXLOVqtSvi7wCz10KBAZg7rwVs3jGPZA1P51Rn92JhXwQ1vrWbqM9/z5o/7qKpvNM/AFa9ASLTHUjFLqo9y3RureOHb3cwalc4nt0+kX3K0axcTAjJOUW9QLpAaG86Ht05gQPdobv3vWhZtOeKaHaD+dot3Q6Kd0IUDfLAml398s5tLRqfza3vhD3t0S1PbNuL4rUmMCmXutWMoq23g7nnraXLGuahxrq3Ct9sLeG9VLrdM7svYzA4yvBzop3P1KT05pXc8f1q4ncJKB/onObhoW9fQxOOfb2NAiirS8zZGxPDTgNZvg3nAiY0g7B2TBhw24P4n888xKtvDEUSAyvsOjoDQaPUVFqP+KCITVWvWmDT1hx7bU30F+mEGjJW02HB+c/YA7jozi0Vbj/Dmj/t4/PNtPLN4F5ePzeCGiZmkx0V4z6DKfNg6H8bNcTks0R4r95Zw17z1lNU28tdLhjE7O8P9j9A9x8OOL6CqAKKdzyKJjwzhvzefwnVvrOKO/63jhStHHUsbdIaaYiU0ic63VPgxp5jff7yZ0/ol8pdLhjn+M4mxCn7lIUjrODI7JDWGP144hAc+2szz3+52/I3FibYKJdVHeeCjTQzsHs29Zznws4hM7LAJnBCCP88axjnPL+Oxz7fy8lUd1DnUlQECQtv/G375+xwOldfx/pzxplTUGyH49v5STnwrd+QYdaAQc1BhH3r2dHExY+bT0Gw5dhsp1ba5ydo7wQLNjeqjaWO9GqPWUAMN1XC0Sn0CKNiqPrbaVt9tBASpsv+ELNUyt/tw6DFCvRGY0UfFRUKCArhgRCoXjEhlQ245byzfx1s/7eetn/YzY2h3bpnc1/MFQwA/v6TSCsfNMfSyzc2Sl7/P4dmvd9ErIZI3rh9rXM+XDKs/k7sSBl/g0iW6hQXzzo3juPGt1dz53noam5q5cGSacxcp3qW2Tgr+ziNV3PqftfRJiuTlq0c7JzzdrGGwikMOnzI7O4NV+8r455LdjOkV51g2VHWhQ33wpZT87uPNVNZZ+O/NpxAa5EBIMCIe8jd0eFifpCjumtqPpxfv4uttBe1n7dSVK0exnU8kOYXVvPrDHi4amXpSzY23MELw84CMVt+nA/kuHAOAlHIuMBcgOzvbtXjLqKtcOs0ulqNQdVj9gZcfUH1LSnKgaCfs/kqJFaiFs57j1bCMvlNVqbufMDIjlheuHMWDMwby1k/7eW/lQb7YdJhT+yQw5/Q+nNE/yTMLS7Wlqg3y0EsN7Yp5pKKe33y4keU5xVwwIpU/zRpmbGpqjxEQGOqW4ANEhwXz1g3juOnt1dzz/gaONjYze2xGxyfaaEnJdDykc7iijhvfWk1YSCBv3uBCVWlkonrtlR2HdGwIIXjyoqFsOVTBPfPW8/mdp3X8KbKmyKEsqP+uOMDibQU8NHMQA7s7+AkxIkE5c1J26KTNmdyXzzce5pFPtzC+T3zbdRT15e0u2KrF8c1EhATx8HnutYtwByP+C1YDWUKI3sAh4ArgFycc8xlwhxBiHircUyGl9Ew4x2iCQpVHH5cJTDx+X2MdFGyD/HVwcIXqAbN1vtoX1xuypsOg86HXBI8tRhpJamw4v585iDun9uO9VQd5Y/l+bnhzNQNSovnl5D6qIvHE6lN3WPEKNNbApF8bdskvNuXz0PwtNFia+fOsYVwx1oAQzokEhapwhpOZOvaIDA3izevHcct/13L/R5uobbBw/UQH3/yKd0NQ+DGvu6PDq49y1WsrqahrZN6c8aTFhjtvsBDQLdUpDx8gPCSQV64ezYUv/cgv31nLR7ed2nZjMCkdCulsza/giQXbOWNAkmqb4CgRCerTfUMNhEa1e2hIUAB/uWQYl7zyE499to1nZo+wf2AHfXT+b20eK/eV8pdZw0hsq7DPC7j93yultAB3AF8B24EPpJRbhRC3CiFutR62ENgL5AD/Bn7l7n19guBwSB8D434Jl74O926Fu9arkFJif1j3Drx9Hjw7GBbeD/nrzbbYIaLDgpkzuS9L75/C05eNQAj4zYcbmfS3Jbzy/R4qag1Y4K2vhFX/goHnudVh0EZx9VHunreeO/63nt6JkSy8exJXjuvpuZS3jFNUWKCxzu1LhYcE8u9rxzB9cAqPfb6NF5fsbune2i7Fu1WGjgMLmxW1jVzz+iryy+t44/qx7tUBxKSrGL6T9EmK4oUrR7HjSCW//XBT26+xrkyFXNsJ6dQctXDn/9YTFxHMM5eNcK6it4Nc/BMZ1TOOO6b046N1eSzY1Iaf2k4fndKaBv60cDvZveKYne3EJzgPYIi7JqVcKKXsL6XsK6V8yvrcq1LKV62PpZTyduv+YVLKNUbc1+cQQqXtjfslXPUB3L8HLn0D0rNVsc7cM+DVSar9b32l2dZ2SEhQAJeOSefLuyfx9o3j6JccxV8X7WD8n7/lkU+3sLeo2vWLr35N9c6Z/Bu3bGxulry36iBnPvMDCzcf5p5pWfzfrafS28hqVntknKJEyaA38dCgQF66ajQXjUzl6cW7eOCjTTRY2s51B1QM34EK24q6Rm54axV7Cqv51zXZjOvtZp+ibmlOe/g2pgxI5nczBrJg82FeXJJj/6Dq9ouubOGR/SU1/OPyUW23wmiLNtortMedZ2YxIiOW38/fzOEKO2/ydeV2QzpSSp74YhtV9Rb+NGuY6R1v/bzblo8TEglDL4Er3oXf7FKePxIW3AfPDYHFDzuU3mY2QghO75/EuzePZ+Fdkzh3eA/mrcpl6jM/cM3rK/lq6xEsTR2IU2saatVibd8z3WqjsGJvCZe8+hO/+3gzA7tH8+Xdk7hnWn/vNJGzFWAZOAErODCAZ2eP5K6p/fhgTR7XvbGq7U9TlqNqTamD+H1uaS2XvPITm/IqeOHKUY4tmHZETJpa12p2cMjQCfxyUh9mjUrjma938e7KAycf0E6VrZTS2ok0n3um9efUvi4sftoEv+7k9gptERwYwPOXj6SxqZn7PrBTkFVXZjek89ZP+5m//hC/mtKP/ikupgIbiBZ8bxEeqzz/W5bBzUvUcI+fX4bnR8D829RisB8wOLUbT182gh8fnMp9Z/Unp7CaW/6zlol/XcJfF+0gp9ABr3/NG2rRbPJvXbJh/cEyrnl9JVfMXUF+eR1PXzaCeXPGu55b7wqRierTXO4qQy8bEKD60Tw7ewRrD5Rx3ovL+HmPHU+0dK+1aVrbGTprD5Ry0Us/UlR1lHduGsc5Qw0akNItDWSTqm1xASEEf5o1jKkDk3lo/hbmLj3hb7+lj87Jgv/04p28vnwf10/I5M6pLhac2Tqx2umn0x6ZiZE8ev5gftpTwkOfbD5WV9DSGjn2uOO/3V7AE19sY/rgFO4503PTyJxB99LxNkKouP9lb6q2tj+/DGvfhE3vw4grlAj6yhzXdkiKDuXOM7O47Yy+fLujkA9W5zJ36V5e+X4Pw9Nj1Pi4Id3pk3TColhdOSx7GvpMgV6nOny/uoYmPt+Uz7srD7Ixt5z4yBAePncQV4/vRViwSQvi6WNh7/cOZXs4y6zR6fRKiOTXH2zgyn+v4JrxvXhwxsBjMxDaScmsbbDwrx/28soPe+gRE8Yb14+l74m/B3eIsS4SVx46lpfvJGHBgbx69Rju/WADf1q4g+qjTdw7LUutudgJ6TQ3S57/djcvfbeHK8dl8Oj5bvTcdyGkY2N2dgYHS2t56bs91DY08cxlIwiy1KpU71Ye/rb8Su58bz2DU7vxjyvc6BpqMFrwzSS2J8z4C5x2L/z4D+X5bvpA5aRP/o1Xe8K7SlBgAGcP6c7ZQ7pTWFXPp+vz+WJTPn9btJO/LdpJn6RITu2TwPg+CYzNjCdl1XOIunI464/tXldKyZHKepbuKuL7nUUs311M1VELWclRPHr+YC7LzjC/C2j6WPVGXZGrfpcGM6ZXHIvunszTi3fyxo9qqtQVYzO4fGwG6S1zbI95ufWNTSzYdJi/f7WTI5X1nDu8B09cOLT9NgOu0Lra1hbacoGQoABeuGIUkSGBvPDtbr7fWcivz+rP6dWFiIDglkXQn/eU8OSCbWzNr2TWqDSeusiJQjF7hMWogksnPXxQn05+e/ZAIkKC+PtXO6lraOIv0+KJBwiPpa6hiXdXHuDl7/fQLSyY168b691xlx0gHMoGMIns7Gy5Zk3nXN+1S+Vh+O4pWP9fVXV6+oMqDOSHlb355XV8tfUIS3cVsXp/GdVHLaRSzHdh9/Fz2CQ+6/MoseEhxIQHExESSH1jEzUNTVTUNbCnqIbdBVWUWePX3buFccaAJGaNTmdsZpw5g8LtcXgj/GsyXPI6DLvUo7dae0ANv/l+VxEAb8W8znDLJl4d/TkIWHegjI25FTQ0NTMiPYY/nDeY7I5aDLhKXRn8NROmPwkT7nT7cs3Nkv9bl8fz3+zmUHkdr8e+yRjLBv7Q90OKqupZsbeUtNhw7j9nAOcPTzXGW/5rbxhyMZz3rMuXsFWpDxQHWRT6IO/2epLnDg2iuPooE/om8PgFQ8gyIW4vhFgrpbQ759V33no00K0HXPgijL9NLeh+9TtY/x+Y+XfIPM1s65wiNTacGyb25oaJvbE0NbM1v5LIL+8k8DD8X7frWLenhIq6xpZB2KA8vujQIPokRXLO0O5kJUczoV8CA1KifUfkW5M8RLXkyFvtccEf0yuON28YR15ZLe+vzqXHqlxymnvw5k/7aWqWDE2L4fqJmWoCWlaSZ0MIYbEQHOlyps6JBAQIZmdncNHINN5fk0v0N6Ucae7G1kMVhAQF8NuzB3DTab2NDd050E+nI26Y2JtxvePZtXIRbISv9tbTv2cUL/1ilGmVtB2hBd8XSRkCV38MOxfClw/CW+fCsNlw9p8gyoAsCy8TFBjAiOBcyP8cJtzBi9MvatlnsQ5+DwsONKW3iFsEBkHqaEMzdToiPS6C+87qD2uPwKjL2TnzHCzN0rs/OyFU7N6JaltHCAkK4JrxvWCjBaL6suSqMwy9/nFEJDiVpdMWQ1JjGDIwAjbC27edhUgd6b5tHsTP/sO6EEKo3vC3r1QLuVvnw0tjYcP//K/DZ3MzLPiNip2ednxVbVBgANFhwf4n9jYyxsLhTaonk7eoLoCjlZDYHyGEOT87N3LxO6S6yKE+Om4REe9SDN8u1tbIwoUB5t7GT//LuhAhETD1Ybh1ucq5/uQ2+M9FUGYnf9lXWfO6aid89p/8YiHaKdLHqgKswxu8d0/bgq2bYw3dIibNpWrbDmluVnn4Lg4vd5iIeLdDOi04MfzEbLTg+wvJA+GGRXDuM5C3Bl6ZoKpV25k85BOU58I3j6k0zJEntljqBKQbX4DVIS0pma71wTeEbukqfdLSYOx168tViqPHBd8awzfi03Jducr6CTG/sKojtOD7EwEBMPZm+NXPyrNccB+8cwGU7TfbMvtICQt+rQqEzv+HX7WPdpioJNVYz+ACrHYp2qkWTaNdmBVrFDFpgIQqu01vXafaOgXMheHlTtG6gZq71JerhWwHh7WYie9bqDmZ2J5wzXy44J+qgdfLE2D1674X29/0AexeDFP/YO022klJH6s8fG/9/Au3qYZzZgpMSy6+wWGdKmtzsm4efjNzsoFau7TRR8cX0YLvrwgBo69V3n7GWOVJ/+ciVb3rCxRshS/ugYzxcMotZlvjWdLHKaHyREz7RKRUP9sU83qqA8dX2xqJrV1DtEFtINrCjWrbk2ijj44vogXf34nNgGs+gXOfVbH9lyeooSJmevt1ZTDvKjXubfbbfjELwC0yxqqtAf3xO6S6QKUTJnc8HMSjODHb1ilsHn6UlwTfgNTMjoaf+BJa8DsDQsDYm+C2nyBtlPKs/3OxObH95ib46GYlBLPf8byn5gukDFUFWN6I4xdstd7TZA8/NEql2XrCww+LUdlpnsTFBmp26WD4iS+hBb8zEdcLrv3M6u2vhpdPVW2IXWxj6zRSqgrhnG9g5t+g54mz7DspgcGQNkZNPfM0hdvU1mwPH1SmjtEx/Mp87yxGGxnS0R6+xjRs3v7tK6H3ZPjq9/DamZC31rP3bW6Ghb+BFS/DuFtgzA2evZ+v0XM8HNkMR90YCuMIBdsgKgUifaB03wPVtlQd8c6nQjcaqB2HlNZFW98vugIt+J2XmHS4ch5c+qbyml6bCp/cfqz1rJE0WeDT21VdwIS7YMZfO2cKZntknKJ6xB/y8Btr4VZINjmcYyMmQ9VZGEnVEYjuYew17REQqMIw7nr4DdXq994VQjpCiL8LIXYIITYJIeYLIWLbOG6/EGKzEGKDEKILtb80GSFg6Cy4c60S4k3vwz/HwHd/bikHd5vKw/DeFbDxf3DG71Xb464m9qBSMxGeXbhtblI5+Ck+EM4BlTBQXw5Hq4y5XnMzVHvJwwdDGqhRV6a2XSSk8zUwVEo5HNgF/K6dY6dIKUe21bZT40FCo2H6EyqFs/dk+OEv8Pxw+OFvUFPs2jWlhA3vwcunwP5lanzjGQ90TbEH9Q+fPMizgl+6Fyz1vuXhg3Fefm2JqrL1hocPBgl+udp2BQ9fSrlYSmmxfrsCSHffJI3HSMxS83VvWQq9TlO9958ZCB9eD3uWQFMb81NbYzkKWz5SHTw/uRWSBsGtP6q+/V2djFMgd7Xn2l3YMnSSB3nm+s5iG/pSYZDg26p2u3lR8G0euqv4UR8dMLY98o3A+23sk8BiIYQE/iWlnNvWRYQQc4A5AD17Gj9FSAP0GAFX/g8Kt8Pat2HTPNWNMzhSZdb0mgCxmeqPOCxGfQooyYHinbBjocpdjukJM/6mWj109jx7R+k5Xo2rLNrumbBL4TZAQNJA46/tCi0evkHFfi1FV94S/DjIX+feNfzMw+9Q8IUQ3wD2gmoPSSk/tR7zEGAB3m3jMhOllPlCiGTgayHEDinlUnsHWt8M5oKaeOXAa9C4SvIgNWJx2mOw+yvYtwwO/AhLnrR/fHg89J4Eo69TzdD8oHeIV7GN+zu4wjOCX7BVDU73dI66o0SlQGCIgR6+tejK2zF8d2YSt3j4/pGl06HgSymntbdfCHEdcB5wpmxjXqKUMt+6LRRCzAfGAXYFX2MCwWEw+EL1BepjbnWh8l7qK9Q/RkIfv/mjNo243qqPe+5KlRprNIXbzS+4ak1AgKq4NSqGX3UEEOqNxBu0bqAW6uKQd5uH3xVCOkKIc4AHgNOllLVtHBMJBEgpq6yPpwPtT7DWmEt4nBZ3VxBChcQ8UYDVUKsWbYddZvy13SE2w1gPPzLJezOcWzdQc1nwy0AEQoiL53sZdz+TvwhEo8I0G4QQrwIIIVKFEAutx6QAy4UQG4FVwAIp5SI376vR+CYZ46H8wLF4tFEU7QCkb3n4oNZyjPLwKw97txWHEdW2tipbP8lOc8vDl1LaHbljDeHMtD7eC4xw5z4ajd/Qc7zaHlwBQy4y7rq+1FKhNbEZKnfechSCQt27VtVh7y3YQivBd6Pa1o/66ICutNVojKX7cJXttH+5sdct2ApBYRDf29jruostU8eIrpneaqtgw9ZAzZ2OmX7URwe04Gs0xhIUAr1OhX0G5yTkrYbUUb6XAhtrE3w3wzpNjVBTZJKH70ZIx4/66IAWfI3GeHqfrmoWKg8bc73GejXZLH2sMdczEqOqbasLAOm9oito1UDNHcH3n+EnoAVfozGe3pPVdv8yY653eCM0Nx7L8/cluqUBwn0P39tFV6A+LYXHuRfD1yEdjaaL03248vr2/mDM9fKsg1XSfVDwg0KUSLvr4Xu76MpGeLzrHn5zs6pT0R6+RtOFCQhQFcn7fjBm1GTuSojtBdFeKkhyFiNy8c3w8MG9BmoNVSCbtYev0XR5ep+uRLBsn3vXkVI1ZPPFcI6NmAz3++lUHYaAIIhINMYmR3GngVpLa2S9aKvRdG36nKG27oZ1KnJVnnuGD4+LjM1Qs23dGaVZeVgNLvd2f6aIONc9/BrreZFJxtnjYbTgazSeIKGfCk+4m55pG4zuixk6NmIyVB97d6qLq7xcZWujdQM1Z6kpsl7Dy59K3EALvkbjCYRQYZ19S93rj5+7CoIjIGWocbYZjRF98b1ddGUjIlE1UDta6fy5NsGP1IKv0Wh6T4baYtUf31XyVkHaGAg0cnSFwRiRi+/ttgo2bJ05q4ucP1cLvkajaaHP6Wq75zvXzm+ohSObfTucA62qbV1cuG2sU/nsZnj4UclqW13g/Lm1JaqNRkiksTZ5EC34Go2niEmHlGGw/TPXzs9fr2LjvpyhA0rwwuNd9/BtOfjdUo2zyVFaPHwXBL+mCCITjLXHw2jB12g8yZCLVB69K83F8vxgwdaGO7n4ldZZtqZ6+IXOn1tT5FcZOqAFX6PxLEMuVtttnzp/7t4fICHLP2LEsW70xS+11irEmdAJNDxeDTCp0YKv0WjcJaGvarWwdb5z59UUqwyfQed7xi6jictUg19cycUv26eKrmyLv94kIEB5+S6FdEr84824FVrwNRpPM+Ri1d7YmWrUbZ+CbIKhszxnl5EkZIGl3rWwTule1TrCrEykqGTnQzpSKg/fj3LwQQu+RuN5bJOvnAnrbJ2vRNSX8+9bk5iltsU5zp9butfcwS6RLgh+fYXqYNqVQjpCiMeEEIes82w3CCFmtnHcOUKInUKIHCHEg+7cU6PxO+L7QI+RsOVjx46vOqImZg2d5TezUknsr7bFu5w7T0oVw4/vY7xNjhKV4rzg1xSrbVcSfCvPSSlHWr8WnrhTCBEIvATMAAYDVwohfGwSs0bjYYZcDPnroGx/x8du+xSQMMRPwjmgWhSExULJbufOqy1VVa6mCn6yWrR1piLaD4uuwDshnXFAjpRyr5SyAZgHXOiF+2o0voMtrLP5w46P3fIxJA+G5IEeNclQhFBefrGTgl+6V23N9vCbLc51zay1efhdT/DvEEJsEkK8IYSw1yc0DWi9kpNnfc4uQog5Qog1Qog1RUUulDtrNL5IXCb0nQo/vdj+hKWKPMhd4V/evY3ELNcF34yUTBtR1rCMM6mZLR5+JwvpCCG+EUJssfN1IfAK0BcYCRwGnrF3CTvPtdmaTko5V0qZLaXMTkryrx+mRtMu059U4YulT7d9zNZP1NZfsnNak5ilWjnXO9GIrHQvICCul8fM6hBXqm1tMXw/y9LpMA9KSjnNkQsJIf4NfGFnVx7QOsE2Hch3yDqNpjORMgRGXQ2r5sLYm1SOfmtqimHFy5A66uR9/kCCNVOnZLdq+OYIpXtV/n1QqOfs6ogWwXfSww+LUSMe/Qh3s3Rat7e7GNhi57DVQJYQorcQIgS4AnCxuYhG4+dMeRgCQ+CbR49/vrkJPrpJif75z5tjm7u4kppZts/clExwrYFaTbHfeffgfgz/b0KIzUKITcAU4F4AIUSqEGIhgJTSAtwBfAVsBz6QUm51874ajX8SnQKn3QvbP4edXx4bvPHdn2Dv93DuM9BjhKkmukxcb9WmwJnUzNK95i7YAoR2g8BQ5z18P4vfgwMhnfaQUl7TxvP5wMxW3y8ETkrZ1Gi6JKfeDuvfgfeuUCKZORHW/xdGXwuj7f5L+QdBIWpx2tHUzLpy1WLYbA9fCOdz8WuK/TLspittNRpvExIBc36A819Q3u3Geaowa8bfzbbMfZxJzbQNeDfbwwfn++l0RQ9fo9G4SEQ8jLlOfdVXqLh+cJjZVrlPYj/Ys0StSQQEtn9sqS8JfopjRXGgXlttiV8KvvbwNRqzCYuB4HCzrTCGxP7QdNSxRnEtOfiZHjXJIaKSHM/DrysDpN8VXYEWfI1GYyQtqZkOZOqU7oOo7r4xIjAqRcXlmywdH+unbRVAC75GozGSltRMBzJ1fCFDx0ZUMiCPtUxoDz+tsgUt+BqNxkgiEiA8zrGF2zKTu2S2xplqWy34Go1Gg0pxTMjqOKTTUKOGl5udkmkj0lZ85UD/rpoSte2ChVcajUZzPIn9oWhn+8fYMmJ8xsN3otq2pggQKtPKz9CCr9FojKXHCJXxYku7tEfhdrX1leIlZwU/IqHjtFMfRAu+RqMxlr5T1Xbvd20fs/d7lY7qKyMcQyIhJPpYfL49/LToCrTgazQao0noqzpg7llif7+UsOc76HOGb3nJUUkOevjFfpmSCVrwNRqN0QgBfafA3qX289qLd0NlHvSZ4n3b2sPRfjq1WvA1Go3mGH2mwNEKyF9/8j6b59/X1wTfwX46OqSj0Wg0rehzBiDsh3X2LIH4vr7RUqE1jnj4lgbV+0gLvkaj0ViJiIfUkScv3FqOwv5lxxZ2fYnIZKgvVza2ha0SNyLBKyYZjRZ8jUbjGfpOhdxVx8+4zV0FjbW+KfjdUtW2Iq/tY2whH1sap5+hBV+j0XiGPlNANsH+5cee27MEAoIg8zTz7GqLxP5q217RmG18Y7yP1A84iRZ8jUbjGTLGQXDk8XH8PUsgfRyEdTPPrrZIsgp+cXuCvxNEgO8UjDmJWwNQhBDvAwOs38YC5VLKkXaO2w9UAU2ARUqZ7c59NRqNHxAUqjz5XV/BgBkq7n14I0z5vdmW2ScsRrVrLmqn02fxLrXYHBTqNbOMxN2ZtpfbHgshngEq2jl8ipTSgd6jGo2m0zDofNj9Ffx31rHnfDF+byOpf/seftEuSBzQ9n4fx5ARh0IIAcwGfPg3qdFovM7oa6DfNNX7vmwfNDVC2hizrWqbxAFqxrCUqoCsNU0W1QW0/3RzbDMAo2baTgIKpJRtNcGWwGIhhAT+JaWc29aFhBBzgDkAPXv2NMg8jUZjGt16qK/MiWZb0jFJA6ChSrVutmXt2Cg/AM2NxxZ3/ZAOBV8I8Q3Q3c6uh6SUn1ofXwm8185lJkop84UQycDXQogdUsql9g60vhnMBcjOzpYd2afRaDSG0TpT50TBt2XvdOaQjpRyWnv7hRBBwCygzc9pUsp867ZQCDEfGAfYFXyNRqMxjSSrmBfvOrn1gy22bxvj6IcYkZY5DdghpbRbrSCEiBRCRNseA9OBLQbcV6PRaIwlKgVCY+zn4hfvVvvDY71ullEYIfhXcEI4RwiRKoRYaP02BVguhNgIrAIWSCkXGXBfjUajMRYhrJk6dlIzi3b6dfweDFi0lVJeb+e5fGCm9fFeYIS799FoNBqvkDgAdi8+/jkp1ZvA8Nnm2GQQutJWo9FoWpPUX41orCs79lx1ARyt9HsPXwu+RqPRtMaWhVPcKsu8JUNHC75Go9F0HpLsNFGzxfST/DclE7TgazQazfHE9oLA0ONbLBTvUkPOo3uYZ5cBaMHXaDSa1gQEqlz71k3Uinaq505st+BnaMHXaDSaE0nsf7KH7+fhHNCCr9FoNCeTNADKDsCRzVBTonrr+PmCLRjXPE2j0Wg6D6mjAAmvngZYwzha8DUajaYT0v9suO1nKNwGRTugphj6nG62VW6jBV+j0WjskTJYfXUidAxfo9Fougha8DUajaaLoAVfo9Fougha8DUajaaLoAVfo9Fougha8DUajaaLoAVfo9Fougha8DUajaaLIKSUZtvQJkKIIuCAi6cnAsUGmuMP6Nfc+elqrxf0a3aWXlLKJHs7fFrw3UEIsUZKmW22Hd5Ev+bOT1d7vaBfs5HokI5Go9F0EbTgazQaTRehMwv+XLMNMAH9mjs/Xe31gn7NhtFpY/gajUajOZ7O7OFrNBqNphVa8DUajaaL0OkEXwhxjhBipxAiRwjxoNn2eAMhxBtCiEIhxBazbfEGQogMIcR3QojtQoitQoi7zbbJ0wghwoQQq4QQG62v+XGzbfIWQohAIcR6IcQXZtviDYQQ+4UQm4UQG4QQawy9dmeK4QshAoFdwFlAHrAauFJKuc1UwzyMEGIyUA28I6UcarY9nkYI0QPoIaVcJ4SIBtYCF3Xm37MQQgCRUspqIUQwsBy4W0q5wmTTPI4Q4tdANtBNSnme2fZ4GiHEfiBbSml4sVln8/DHATlSyr1SygZgHnChyTZ5HCnlUqDUbDu8hZTysJRynfVxFbAdSDPXKs8iFdXWb4OtX53HW2sDIUQ6cC7wmtm2dAY6m+CnAbmtvs+jkwtBV0cIkQmMAlaabIrHsYY2NgCFwNdSyk7/moF/APcDzSbb4U0ksFgIsVYIMcfIC3c2wRd2nuv0XlBXRQgRBXwE3COlrDTbHk8jpWySUo4E0oFxQohOHb4TQpwHFEop15pti5eZKKUcDcwAbreGbA2hswl+HpDR6vt0IN8kWzQexBrH/gh4V0r5sdn2eBMpZTnwPXCOuZZ4nInABdaY9jxgqhDiv+aa5HmklPnWbSEwHxWqNoTOJvirgSwhRG8hRAhwBfCZyTZpDMa6gPk6sF1K+azZ9ngDIUSSECLW+jgcmAbsMNUoDyOl/J2UMl1KmYn6X14ipbzaZLM8ihAi0pqIgBAiEpgOGJZ916kEX0ppAe4AvkIt5H0gpdxqrlWeRwjxHvAzMEAIkSeEuMlsmzzMROAalMe3wfo102yjPEwP4DshxCaUY/O1lLJLpCl2MVKA5UKIjcAqYIGUcpFRF+9UaZkajUajaZtO5eFrNBqNpm204Gs0Gk0XQQu+RqPRdBG04Gs0Gk0XQQu+RqPRdBG04Gs0Gk0XQQu+RqPRdBH+H2CPrSMV0KsfAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = torch.linspace(0, 5, 100, requires_grad=True)\n",
    "y = (x**2).cos()\n",
    "y.sum().backward()   # populates the grad attribute below.\n",
    "print(x.grad)\n",
    "\n",
    "plt.plot(x.detach(), y.detach(), label='y')\n",
    "plt.plot(x.detach(), x.grad, label='dy/dx')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise\n",
    "\n",
    "1. Introduce a new vector x<sub>2</sub> which also ranges from zero to five (same as x, but not cloned from x).\n",
    "2. Plot the polynomial y=x<sub>2</sub><sup>3</sup>-6x<sup>2</sup>+8x\n",
    "3. Plot dy/dx and dy/dx<sub>2</sub>.\n",
    "\n",
    "Which x contributes most to the gradient at zero?  At five?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO: define x2 just like x, but not cloned from x\n",
    "x2 = 'TODO'\n",
    "\n",
    "# TODO: Plot the given polynomial which depends on both x and x2\n",
    "\n",
    "# TODO: Plot both dy/dx and dy/dx2.  Explain what you get."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Accumulating and Zeroing grad\n",
    "-----------------------------\n",
    "\n",
    "**Gradient accumulation.** If you find that your data batches are too large to get gradients of the\n",
    "whole thing, then it is usually possible to split the batches into smaller pieces and add the\n",
    "gradients. Because gradient accumulation is a common pattern, if you call `.backward()` when parameters\n",
    "`x.grad` already exists, it is not an error.  The new gradient will be *added* to the old one.\n",
    "\n",
    "**zero_grad().** That means that you need to set any previous value of `x.grad` to zero before\n",
    "running `backward()`, or else the new gradient will be added to the old one.  Optimizers have a\n",
    "utility `optim.zero_grad()` to do this to all the optimized parameters at once."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Saving memory on inference\n",
    "--------------------------\n",
    "\n",
    "**Avoid autograd when you don't need it.** Normally, all the parameters of a neural network are set to `requires_grad=True` by default, so they are ready to be trained.  But that means that whenever you run a network, you will get output which is also requires-grad, and it will be attached to a long computation history that consumes a lot of precious GPU memory.\n",
    "\n",
    "To avoid all this expense when you have no intention of training the network, you could go through all the network parameters to set `requires_grad=False`.\n",
    "\n",
    "Another way to avoid the computation history is to enclose the entire computation within a `with torch.no_grad():` block.  This will suppress all the autograd mechanics (which means, of course, `.backward()` will not function).\n",
    "\n",
    "Note that this is different from the role of `net.eval()` which puts puts the network in inference mode computationally (batchnorm, dropout, and other operations behave differently in training and inference); `net.eval()` does not have any effect on `requires_grad`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "More tricks\n",
    "-----------\n",
    "\n",
    "**Gradients over intermediate values.** Normally gradients with respect to intermediate values are not stored in `.grad` - just original input variables - but you can ask for intermediate gradients to be stored using `v.retain_grad()`.\n",
    "\n",
    "**Second derivatives.** If you want higher-order derivatives, then you want pytorch to build the computation graph when it is computing the gradient itself, so this graph can be differentiated again.  To do this, use the `create_graph=True` option on the `grad` or `backward` methods.\n",
    "\n",
    "**Gradients of more than one objective.** Usually you only need to call `y.backward()` once per computation tree, and pytorch will not let you call it again. To save memory, pytorch will have deallocated the computation graph after you have computed a single gradient.  But if you need more than one gradient (e.g., if you have different objectives that you want to apply to different subsets of parameters, as with happens with GANs sometimes), you can use `retain_graph=True`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise\n",
    "\n",
    "1. Plot the polynomial y=x<sup>3</sup>-6x<sup>2</sup>+8x, just as in the first exercise.\n",
    "2. Use `y.sum().backward(create_graph=True)` to compute the gradient, and plot dy/dx.  Why is `x.grad.detach()` needed now?\n",
    "3. Now set `dy = x.grad.clone()` and then `x.grad.zero_()`, before using `dy.sum().backward()` to compute a 2nd gradient.  Plot d<sup>2</sup>y/dx."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO: define the polynomial just like the first exercise\n",
    "y = 'TODO'\n",
    "\n",
    "# TODO: Use `backward(create_graph=True)` to compute dy/dx.  Plot it.\n",
    "\n",
    "# TODO: Use `backward()` a second time to compute the second derivative.  Plot it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [On to topic 3: Optimizers &rightarrow;](3-Pytorch-Optimizers.ipynb)"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
